Point cloud clustering method and device, equipment and storage medium

Abstract

The invention discloses a point cloud clustering method and device, equipment and a storage medium. The method comprises the steps that at least one space point is mapped to a preset grid system according to point cloud information detected by a vehicle under the ith frame, a space coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix are established, the point cloud information comprises point information of the at least one space point, and the point cloud information comprises point information of the at least one space point; the point information comprises coordinate information, Doppler velocity and a radar cross section; the space coordinate grid matrix, the Doppler velocity grid matrix and the radar cross section grid matrix are processed, the cluster identification number of each space point is determined, at least one space point is clustered into at least one point cloud cluster, and the cluster identification numbers of the space points in different point cloud clusters are different. According to the embodiment of the invention, different targets close to each other in a physical space can be effectively distinguished, and the accuracy of point cloud clustering is improved.

Description

Technical Field

[0001] This application belongs to the technical field of vehicles, and particularly relates to a point cloud clustering method, device, equipment and storage medium. Background Art

[0002] Radar sensors have been playing an increasingly important role in the perception field of intelligent driving due to their characteristics such as being less affected by weather, being able to work all-weather, having the ability to perceive in the longitudinal dimension, and being able to perceive non-line-of-sight targets. The clustering algorithm is the main point cloud information analysis algorithm for millimeter-wave radars and can be used to segment the point cloud information of different individual targets. However, existing point cloud clustering methods may cluster adjacent different point clouds into the same class, that is, different targets are wrongly merged, resulting in low accuracy of point cloud clustering. Summary of the Invention

[0003] Embodiments of this application provide a point cloud clustering method, device, equipment and storage medium, which can effectively distinguish different targets close in physical space and improve the accuracy of point cloud clustering.

[0004] In a first aspect, embodiments of this application provide a point cloud clustering method, and the method includes:

[0005] Obtain the point cloud information detected by the vehicle in the i-th frame, where the point cloud information includes the point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity and radar cross-section;

[0006] According to the coordinate information, Doppler velocity and radar cross-section of at least one spatial point in the point cloud information, map the at least one spatial point to a preset grid system associated with the vehicle body coordinate system of the vehicle, establish a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross-section grid matrix. The preset grid system includes a plurality of grids. The spatial coordinate grid matrix includes the values of a plurality of first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes the values of a plurality of second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross-section grid matrix includes the values of a plurality of third elements, and the value of each third element is the average radar cross-section of the spatial points in the corresponding grid;

[0007] Process the spatial coordinate grid matrix, the Doppler velocity grid matrix and the radar cross-section grid matrix to determine the cluster identification number of each spatial point;

[0008] According to the cluster identification number of each spatial point, cluster the at least one spatial point into at least one point cloud cluster, and the cluster identification numbers of the spatial points in different point cloud clusters are different.

[0009] Second aspect, an embodiment of the present application provides a point cloud clustering device, which includes:

[0010] A first acquisition module, configured to acquire point cloud information detected by the vehicle in the i-th frame, where the point cloud information includes point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross section;

[0011] A building module, configured to map the at least one spatial point to a preset grid system associated with the vehicle's body coordinate system according to the coordinate information, Doppler velocity, and radar cross section of at least one spatial point in the point cloud information, and build a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross section grid matrix. The preset grid system includes a plurality of grids. The spatial coordinate grid matrix includes values of a plurality of first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes values of a plurality of second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross section grid matrix includes values of a plurality of third elements, and the value of each third element is the average radar cross section of the spatial points in the corresponding grid;

[0012] A first determination module, configured to process the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross section grid matrix to determine the cluster identification numbers of the respective spatial points;

[0013] A clustering module, configured to cluster the at least one spatial point into at least one point cloud cluster according to the cluster identification numbers of the respective spatial points, and the cluster identification numbers of the spatial points in different point cloud clusters are different.

[0014] Third aspect, an embodiment of the present application provides an electronic device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the above-mentioned point cloud clustering method as described in any one of the above is implemented.

[0015] Fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned point cloud clustering method as described in any one of the above is implemented.

[0016] Fifth aspect, an embodiment of the present application provides a computer program product, and when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the above-mentioned point cloud clustering method as described in any one of the above.

[0017] Sixth aspect, embodiments of the present application provide a vehicle, including at least one of the following: the point cloud clustering device as described above; the electronic device as described above; the computer-readable storage medium as described above; the computer program product as described above.

[0018] The point cloud clustering method, device, equipment and storage medium of the embodiments of the present application can map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the point cloud information detected by the vehicle in the i-th frame, establish a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross-section grid matrix. The point cloud information includes point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity and radar cross-section. The preset grid system includes a plurality of grids. The spatial coordinate grid matrix includes the values of a plurality of first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes the values of a plurality of second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross-section grid matrix includes the values of a plurality of third elements, and the value of each third element is the average radar cross-section of the spatial points in the corresponding grid. Process the spatial coordinate grid matrix, the Doppler velocity grid matrix and the radar cross-section grid matrix to determine the cluster identification numbers of each spatial point. According to the cluster identification numbers of each spatial point, cluster at least one spatial point into at least one point cloud cluster, and the cluster identification numbers of the spatial points in different point cloud clusters are different. In this way, in the embodiments of the present application, when performing point cloud clustering according to the coordinate information of at least one spatial point, the Doppler velocity and radar cross-section of at least one spatial point are also introduced to participate in the clustering. In this way, through the average Doppler velocity in the Doppler velocity grid matrix and the average radar cross-section in the radar cross-section grid matrix, the negative impact of noise points on the clustering result can be reduced, thereby improving the robustness of the algorithm, effectively distinguishing different targets close in physical space, and improving the accuracy of point cloud clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

[0020] Figure 1 is a schematic flowchart of the point cloud clustering method provided by the embodiments of the present application;

[0021] Figure 2 is a schematic flowchart of the basic process of point cloud data processing provided by the embodiments of the present application;

[0022] Figure 3 is a schematic flowchart of the dynamic point cloud multi-dimensional wavelet clustering provided by the embodiments of the present application;

[0023] Figure 4 It is a schematic flowchart of static point cloud multi-dimensional wavelet clustering provided by an embodiment of the present application;

[0024] Figure 5 It is a schematic flowchart of a fast association method based on a wavelet clustering grid provided by an embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of the clustering effect of a multi-dimensional wavelet clustering algorithm provided by an embodiment of the present application;

[0026] Figure 7 It is a schematic structural diagram of a point cloud clustering device provided by another embodiment of the present application;

[0027] Figure 8 It is a schematic structural diagram of an electronic device provided by still another embodiment of the present application. Detailed implementation manners

[0028] The features and exemplary embodiments of various aspects of the present application will be described in detail below. For the purpose of making the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0029] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article or device including the said elements.

[0030] Radar sensors are playing an increasingly important role in the perception field of intelligent driving due to their characteristics such as being less affected by weather, being able to work all-weather, having the ability to sense in the longitudinal dimension, and being able to sense non-line-of-sight targets. The clustering algorithm is the main point cloud information analysis algorithm for millimeter-wave radars and can be used to segment the point cloud information of different individual targets. However, existing point cloud clustering methods may cluster adjacent different point clouds into the same class, that is, different targets are wrongly merged, resulting in low accuracy of point cloud clustering.

[0031] To solve the problems of the existing technology, the embodiments of the present application provide a point cloud clustering method, device, equipment and storage medium. First, the point cloud clustering method provided by the embodiments of the present application will be introduced below.

[0032] Figure 1 The flowchart of the point cloud clustering method provided by an embodiment of the present application is shown. As Figure 1 shown, a point cloud clustering method may include the following steps S101 to S104:

[0033] S101. Obtain the point cloud information detected by the vehicle in the i-th frame. The point cloud information includes the point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross-section;

[0034] S102. According to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the point cloud information, map at least one spatial point to a preset grid system associated with the vehicle's body coordinate system, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix. The preset grid system includes multiple grids. The spatial coordinate grid matrix includes the values of multiple first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes the values of multiple second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross-section grid matrix includes the values of multiple third elements, and the value of each third element is the average radar cross-section of the spatial points in the corresponding grid;

[0035] S103. Process the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross-section grid matrix to determine the cluster identification number of each spatial point;

[0036] S104. According to the cluster identification numbers of each spatial point, cluster at least one spatial point into at least one point cloud cluster, and the cluster identification numbers of the spatial points in different point cloud clusters are different.

[0037] The point cloud clustering method according to the embodiments of the present application can map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the point cloud information detected by the vehicle in the i-th frame, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix. The point cloud information includes the point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross-section. The preset grid system includes a plurality of grids. The spatial coordinate grid matrix includes the values of a plurality of first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes the values of a plurality of second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross-section grid matrix includes the values of a plurality of third elements, and the value of each third element is the average radar cross-section of the spatial points in the corresponding grid. Process the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross-section grid matrix to determine the cluster identification number of each spatial point. According to the cluster identification number of each spatial point, cluster at least one spatial point into at least one point cloud cluster, and the cluster identification numbers of the spatial points in different point cloud clusters are different. In this way, in the embodiments of the present application, when performing point cloud clustering according to the coordinate information of at least one spatial point, the Doppler velocity and radar cross-section of at least one spatial point are also introduced to participate in the clustering. In this way, through the average Doppler velocity in the Doppler velocity grid matrix and the average radar cross-section in the radar cross-section grid matrix, the negative impact of noise points on the clustering result can be reduced, thereby improving the robustness of the algorithm, effectively distinguishing different targets close in physical space, and improving the accuracy of point cloud clustering.

[0038] In S101, the above-mentioned point cloud information includes the point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross-section. The point information is not limited to this, and may also include basic information such as radial distance Range, elevation angle Elevation, azimuth angle Azimuth, phase Phase, and timestamp Timestamp.

[0039] The above-mentioned coordinate information can be used to represent the position coordinates of the spatial point in the vehicle body coordinate system. Exemplarily, it can be the x, y, and z values of each spatial point in the Cartesian coordinate system. Among them, for the position coordinates (x, y, z), the x value can be calculated by Range*Azimuth, the z value can be calculated by Range*Elevation, and finally calculate sqrt(Range 2 –x 2 –z 2 ) to obtain the y value.

[0040] The above Doppler velocity refers to the velocity measured by the Doppler effect caused by the velocity of an object relative to an observer. The Doppler effect means that when an object approaches or moves away from an observer, the frequency of the wave it emits will change. In sensor technologies such as radar and millimeter-wave radar, the Doppler velocity is used to measure the moving velocity of a target object relative to the sensor. By analyzing the frequency change of the received signal, the Doppler velocity of the target object can be calculated.

[0041] The above radar cross section is a physical quantity that describes the ability of an object to reflect signals under radar detection. It is usually used to measure the reflection intensity of an object to radar waves and can be imagined as an area. When the RCS of an object is larger, the reflection signal of it on the radar is stronger, and thus it is easier to be detected by the radar.

[0042] The detection of the above point cloud information, exemplarily, can be devices such as lidar, RGB camera, depth camera, and millimeter-wave radar, etc. In the embodiments of the present application, 4D millimeter-wave radar is specifically used to detect point cloud information. In the embodiments of the present application, it is not limited thereto, and it can also be other devices capable of collecting point cloud information. The selection of different devices depends on specific application requirements and budget limitations, and no specific limitation is made here.

[0043] The above obtaining the point cloud information detected by the vehicle in the i-th frame, exemplarily, can be to parse the original UDP (User Datagram Protocol) data according to the 4D millimeter-wave radar original UDP data packet structure to obtain basic information such as relative radial distance Range, elevation angle Elevation, azimuth angle Azimuth, Doppler velocity Doppler, radar cross section (Radar Cross Section, RCS), phase Phase, timestamp Timestamp, etc., and then calculate the x, y, z values of each point in the spatial coordinate system according to the Range, Elevation, and Azimuth information.

[0044] In S102, the above preset grid system can include multiple grids with equal grid sizes. The number of grids in the preset grid system can determine the value of the GridNum parameter according to the corresponding resolution parameter of the used 4D millimeter-wave radar, and the grid side length parameter GridSide in the x, y, z directions can be obtained by dividing the detection distance value of the radar in each direction by GridNum.

[0045] The above spatial coordinate grid matrix includes the numerical values of multiple first elements, and the numerical value of each first element is the number of spatial points in the corresponding grid.

[0046] The above Doppler velocity grid matrix includes the numerical values of a plurality of second elements, and the numerical value of each second element is the average Doppler velocity of the spatial points in the corresponding grid.

[0047] The above radar cross-section grid matrix includes the numerical values of a plurality of third elements, and the numerical value of each third element is the average radar cross-section of the spatial points in the corresponding grid.

[0048] Based on the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the point cloud information, the at least one spatial point is mapped into a preset grid system associated with the vehicle body coordinate system to establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix. Exemplarily, each spatial point can be mapped into a preset grid system associated with the vehicle body coordinate system according to the magnitudes of x, y, z, and GridSide values to obtain the spatial coordinate grid matrix, and the numerical value of each first element in this matrix is the total number of spatial points PointNum falling within the grid. At the same time, two matrices of the same size as the spatial coordinate grid matrix are established, and the average Doppler velocity and the average RCS are calculated and filled into the elements of these two matrices respectively to generate the Doppler velocity grid matrix and the radar cross-section grid matrix.

[0049] In S103, the above cluster identification number can be used to indicate the cluster corresponding to each spatial point, and the cluster identification numbers of spatial points in different clusters are different.

[0050] The above processing of the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross-section grid matrix is performed to determine the cluster identification numbers of each spatial point. Exemplarily, it can be to perform dilation processing on the Doppler velocity grid matrix and the radar cross-section grid matrix to determine the Doppler velocity and radar cross-section of the associated grids of each target grid in the Doppler velocity grid matrix and the radar cross-section grid matrix, obtaining the dilated Doppler velocity grid matrix and the dilated radar cross-section grid matrix, where the target grid is the grid corresponding to the non-zero numerical value in the Doppler velocity grid matrix or the radar cross-section grid matrix; perform wavelet transform on the spatial coordinate grid matrix to obtain a wavelet matrix; set the numerical values of the elements in the wavelet matrix that are less than the preset wave threshold to zero values to filter and obtain a filtered matrix; in the case where the Doppler velocity difference between the target element and its associated elements corresponding to the dilated Doppler velocity grid matrix in the filtered matrix is less than the preset Doppler velocity threshold, and the radar cross-section difference between the target element and its associated elements corresponding to the dilated radar cross-section grid matrix is less than the preset radar cross-section threshold, the target element and its associated elements are determined to be in the same clustering cluster to obtain a connected domain matrix, where the target element is any non-zero numerical element in the filtered matrix, and the associated elements are the non-zero numerical elements in the filtered matrix that are connected to the target element; assign cluster identification numbers to each clustering cluster in the connected domain matrix to obtain a labeled matrix, where different clustering clusters correspond to different cluster identification numbers; map the labeled matrix into the preset grid system to determine the cluster identification numbers of each spatial point.

[0051] In S104, the cluster identification numbers of the spatial points in the above different point cloud clusters are different.

[0052] The above-mentioned clustering of at least one spatial point into at least one point cloud cluster according to the cluster identification numbers of each spatial point. Exemplarily, it can be the distribution of at least one spatial point according to the cluster identification numbers of each spatial point, and the spatial points with the same cluster identification number are assigned to the same point cloud cluster, and at least one point cloud cluster can be obtained.

[0053] In some embodiments, the above S103 may specifically include:

[0054] Perform dilation processing on the Doppler velocity grid matrix and the radar cross-section grid matrix to determine the Doppler velocity and radar cross-section of the associated grids of each target grid in the Doppler velocity grid matrix and the radar cross-section grid matrix, and obtain the dilated Doppler velocity grid matrix and the dilated radar cross-section grid matrix. The target grid is the grid corresponding to the non-zero value in the Doppler velocity grid matrix or the radar cross-section grid matrix;

[0055] Perform wavelet transform on the spatial coordinate grid matrix to obtain a wavelet matrix;

[0056] Set the values of the elements in the wavelet matrix that are less than the preset wave threshold to zero values, and filter to obtain a filtered matrix;

[0057] In the case where the Doppler velocity difference between the target element and its associated elements in the dilated Doppler velocity grid matrix in the filtered matrix is less than the preset Doppler velocity threshold, and the radar cross-section difference between the target element and its associated elements in the dilated radar cross-section grid matrix is less than the preset radar cross-section threshold, the target element and its associated elements are determined as the same clustering cluster to obtain a connected domain matrix. The target element is any non-zero value element in the filtered matrix, and the associated element is a non-zero value element that is connected to the target element in the filtered matrix;

[0058] Assign cluster identification numbers to each clustering cluster in the connected domain matrix to obtain a labeled matrix, and different clustering clusters correspond to different cluster identification numbers;

[0059] Map the labeled matrix to a preset grid system to determine the cluster identification numbers of each spatial point.

[0060] The above-mentioned performing wavelet transform on the spatial coordinate grid matrix to obtain a wavelet matrix. Exemplarily, it can be to perform convolution calculation after filling the spatial coordinate grid matrix, perform one-dimensional column convolution and one-dimensional row convolution in sequence according to the used wavelet basis function, or directly perform two-dimensional convolution to obtain a wavelet matrix.

[0061] The above-mentioned method sets the values of the elements in the wavelet matrix that are less than the preset wave threshold to zero values, and filters to obtain a filtered matrix. Exemplarily, it can be setting the values of the elements in the wavelet matrix that are less than the preset wave threshold to zero values, and keeping the values of the elements greater than or equal to the preset wave threshold unchanged, and filtering to obtain a filtered matrix.

[0062] The above-mentioned connected component matrix, exemplarily, can use 0-1 to represent the relationship of connectivity and belonging to the same cluster among the elements in the matrix. 1 indicates that multiple elements are connected and belong to the same cluster, and 0 indicates that two elements are not connected and irrelevant.

[0063] The above-mentioned clustering cluster is composed of target elements and their associated elements in the filtered matrix. Among them, the target element is any element with a non-zero value in the filtered matrix, and the associated element is an element with a non-zero value that is connected to the target element in the filtered matrix. Exemplarily, the associated element is an element with a non-zero value within 8 directions adjacent to the target element, and the Doppler velocity difference corresponding to the target element and its associated elements in the dilated Doppler velocity grid matrix is less than the preset Doppler velocity threshold, and the radar cross-section difference corresponding to the target element and its associated elements in the dilated radar cross-section grid matrix is less than the preset radar cross-section threshold.

[0064] The above-mentioned cluster identification number, exemplarily, can be from 0 to 100. Different clustering clusters correspond to different cluster identification numbers. The number of clustering clusters depends on the number of point clouds detected in the i-th frame. Since the amount of point cloud data input for each frame is not fixed, the number of clustering clusters is positively correlated with the number of point clouds input in the current frame and can increase as the number of point clouds increases.

[0065] In this embodiment, a wavelet matrix is obtained by performing a wavelet transform on a spatial coordinate grid matrix, and then a filtered matrix is obtained by filtering the wavelet matrix. Then, according to the dilated Doppler velocity grid matrix and the dilated radar cross-section grid matrix, the target elements and their associated elements in the filtered matrix are determined to be the same clustering cluster to obtain a connected component matrix, and cluster identification numbers are assigned to each clustering cluster in the connected component matrix to obtain a labeled matrix. Finally, the labeled matrix is mapped to a preset grid system, and the cluster identification numbers of each spatial point can be accurately obtained.

[0066] As an implementation manner of the present application, in order to effectively verify different targets, after mapping the labeled matrix to the preset grid system and determining the cluster identification numbers of each spatial point, the above method may further include:

[0067] According to the connected component matrix of the i-th frame and the connected component matrix of the (i-1)-th frame, an intersection matrix is calculated. The intersection matrix includes a first value and a second value. The first value indicates that any element in the connected component matrix of the i-th frame and the corresponding element in the connected component matrix of the (i-1)-th frame are both not 0, and the second value indicates that at least one of any element in the connected component matrix of the i-th frame and the corresponding element in the connected component matrix of the (i-1)-th frame is 0;

[0068] According to the intersection matrix, calculate the correlation ratio of each cluster in the label matrix of the i-th frame. Each correlation ratio is the ratio of the number of grids of each cluster in the intersection matrix to the number of grids of each cluster in the label matrix;

[0069] When the correlation ratio is greater than or equal to a preset threshold, determine that the cluster in the i-th frame and the corresponding cluster in the (i - 1)-th frame are associated clusters;

[0070] When the correlation ratio is less than the preset threshold, determine that the cluster in the i-th frame and the corresponding cluster in the (i - 1)-th frame are not associated clusters.

[0071] The above intersection matrix includes a first value and a second value. Exemplarily, the first value can be 1, and the second value can be 0. The first value can be used to indicate that any element in the connected component matrix of the i-th frame and the corresponding element in the connected component matrix of the (i - 1)-th frame are both not 0; the second value can be used to indicate that at least one of any element in the connected component matrix of the i-th frame and the corresponding element in the connected component matrix of the (i - 1)-th frame is 0.

[0072] The above correlation ratios can be the ratios of the number of grids of each cluster in the intersection matrix to the number of grids of each cluster in the label matrix.

[0073] In this embodiment, after mapping the label matrix to a preset grid system and determining the cluster identification numbers of each spatial point, an intersection matrix can also be calculated based on the connected component matrix of the i-th frame and the connected component matrix of the (i - 1)-th frame. By comparing the correlation ratio with the preset threshold, the association relationship between the clusters in the i-th frame and the corresponding clusters in the (i - 1)-th frame can be judged, thereby effectively verifying different targets.

[0074] As another implementation manner of the present application, in order to effectively distinguish different targets with different movements and stillness that are close to each other in physical space, before the above S102, the method may further include:

[0075] Obtain the Doppler velocity thresholds of at least one spatial point;

[0076] Determine the spatial points with Doppler velocity less than the Doppler velocity threshold as static points;

[0077] Determine the spatial points with Doppler velocity greater than or equal to the Doppler velocity threshold as dynamic points;

[0078] The above S102 may specifically include:

[0079] According to the coordinate information, Doppler velocity, and radar cross-section of at least one dynamic point, map at least one dynamic point to a preset grid system associated with the vehicle body coordinate system to establish a first sub-space coordinate grid matrix, a Doppler velocity grid matrix, and a first sub-radar cross-section grid matrix;

[0080] Map at least one static point to a preset grid system according to the coordinate information and radar cross section of at least one static point, and establish a second subspace coordinate grid matrix and a second sub-radar cross section grid matrix;

[0081] The space coordinate grid matrix includes a first subspace coordinate grid matrix and a second subspace coordinate grid matrix, and the radar cross section grid matrix includes a first sub-radar cross section grid matrix and a second sub-radar cross section grid matrix.

[0082] For obtaining the Doppler velocity threshold of at least one space point, exemplarily, the Doppler velocity threshold of at least one space point can be set by the user according to requirements. Or, it can also be based on the cosine value cos(Azimuth) of the azimuth angle. When cos(Azimuth) is greater than 0.8, the Doppler velocity is compensated by dividing it by cos(Azimuth), and then all the data is sorted in ascending order of the Doppler velocity to find the median v m And calculate to obtain the forty percentile v 40 And the sixty percentile v 60 , calculate the base interquartile range IQR = v 60 - v 40 + v Thre1 , where v Thre1 is a preset constant value greater than zero, which is used to ensure that the difference between the upper and lower limits of the determination interval is not zero. Then calculate the upper and lower limits for distinguishing static and dynamic points. The upper limit v upper = v 60 + IQR * v Thre2 , the lower limit v lower = v 40 - IQR * v Thre2 , where v Thre2 is also a preset constant value, which is used to dynamically adjust the upper and lower limit ranges according to the data characteristics. Finally, calculate the difference between the upper and lower limits v diff = v lower - v upper . Set the threshold constant v Thre3 , which is used to determine whether there are too few static targets in the point cloud of the current frame to divide dynamic points and static points according to this algorithm. When v diff is less than v Thre3 , it is considered that the number of static points in the current frame is sufficient and the discrimination basis for static and dynamic points is credible. At this time, the point cloud that satisfies the Doppler velocity being greater than the lower limit v lower and less than the upper limit v upper is determined as a static point, and the rest of the point cloud is determined as a dynamic point. When v diff is greater than v Thre3When it is considered that the number of static points in the current frame is seriously insufficient, the upper and lower limits in the historical frame are used as the criteria for determining static and dynamic points in the current frame to distinguish dynamic points and static points.

[0083] The process of establishing a grid for the above-mentioned static points and dynamic points has little difference, mainly reflected in that compared with dynamic points, the Doppler velocities of static points are all near 0, and it is impossible to distinguish dynamic targets from static targets through the difference in Doppler velocity values. Therefore, only the RCS matrix is used to judge whether the grid elements are connected. Since the RCS of different types of targets is different, different types of targets close to each other in physical space, such as a vehicle and a person beside the vehicle, can be distinguished by this method.

[0084] In this embodiment, the static and dynamic segmentation of at least one spatial point is performed through the Doppler velocity threshold of at least one spatial point, and subsequent clustering calculations are respectively performed on the segmented static points and dynamic points, so that targets with different static and dynamic states close to each other in physical space can be effectively distinguished.

[0085] In some embodiments, before the above-mentioned S102, the above method may further include:

[0086] In the point cloud information, the point information with a radar cross section lower than the preset signal-to-noise ratio threshold is filtered to obtain the first point cloud information;

[0087] The above-mentioned S102 may specifically include:

[0088] According to the coordinate information, Doppler velocity, and radar cross section of at least one spatial point in the first point cloud information, at least one spatial point is mapped to a preset grid system associated with the vehicle body coordinate system to establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross section grid matrix.

[0089] The above-mentioned signal-to-noise ratio threshold may be the ratio of the intensity of the signal to the intensity of the noise set by the user according to actual needs.

[0090] In this embodiment, when the radar cross section is lower than the signal-to-noise ratio threshold, it usually means that the signal intensity of this point is very weak, which may be due to measurement errors caused by noise, occlusion, or other factors. By filtering out the noisy point cloud with a radar cross section lower than the signal-to-noise ratio threshold, reliable and accurate first point cloud information can be obtained, thereby improving the accuracy of point cloud clustering.

[0091] In some embodiments, before the above-mentioned S102, the above method may further include:

[0092] Obtain the ground clutter range of the vehicle in the i-th frame;

[0093] In the point cloud information, the point information indicating that the spatial point is located within the ground clutter range is filtered to obtain the second point cloud information;

[0094] The above-mentioned S102 may specifically include:

[0095] According to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the second point cloud information, map at least one spatial point to a preset grid system associated with the vehicle body coordinate system, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix.

[0096] The above-mentioned ground clutter is a concept in radar signal processing, referring to the noise or interference from the ground in radar signals. When the radar operates, the transmitted signal passes through the atmosphere and interacts with the ground or objects on the ground, generating echo signals. Among these echo signals, in addition to the effective signals from the target objects, there will also be clutter from the ground.

[0097] The above-mentioned obtaining the ground clutter range of the vehicle in the i-th frame. Exemplarily, it can be to determine the ground clutter determination range according to the radar resolution and the ranging distance of the current radar mode, calculate and analyze the point cloud data analysis, and set the ground clutter to appear within the determination range of |x| < x clutter and |y| < y clutter According to the radar installation height, determine the upper and lower limits z of the threshold in the height dimension upper and z lower .

[0098] In this embodiment, the ground clutter has a negative impact on the performance of the radar system and the target detection ability. Therefore, by filtering out the point information indicating that the spatial point is within the ground clutter range in the point cloud information, reliable and accurate second point cloud information can be obtained, thereby improving the accuracy of point cloud clustering.

[0099] In some embodiments, before the above-mentioned S102, the method may further include:

[0100] In the point cloud information, filter out the point information indicating that the spatial point is outside the preset height threshold range to obtain the third point cloud information;

[0101] The above-mentioned S102 may specifically include:

[0102] According to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the third point cloud information, map at least one spatial point to a preset grid system associated with the vehicle body coordinate system, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix.

[0103] The above height threshold range can be the height range that can be safely traversed after confirming the vehicle height information and the radar installation height. For example, if the vehicle height is 3m and a certain margin is reserved, it can be set to filter out all the point clouds higher than 4m from the vehicle. In this way, the point clouds of traversable targets such as bridges are filtered out. Since the radar installation positions on different vehicle models are different and the vehicle heights are also different, the range is not constant.

[0104] In this embodiment, by filtering out the point information indicating that the spatial point is outside the preset height threshold range in the point cloud information, the third point cloud information is obtained, the radar false detection rate is reduced, and thus the accuracy of point cloud clustering is improved.

[0105] To facilitate the understanding of the point cloud clustering method in the embodiments of the present application, the actual application process of this point cloud clustering method will be described.

[0106] The basic process of point cloud data processing in this embodiment is as Figure 2 shown and includes the following steps:

[0107] Step S1, parsing and preprocessing the original point cloud data of the 4D millimeter-wave radar (equivalent to obtaining the point cloud information detected by the vehicle in the i-th frame) The specific implementation method is as follows:

[0108] According to the original UDP data packet structure of the 4D millimeter-wave radar, the original UDP data is parsed to obtain basic information such as the relative radial distance Range, elevation angle Elevation, azimuth angle Azimuth, Doppler velocity Doppler, radar cross section RCS, phase Phase, and timestamp Timestamp. Then, according to the Range, Elevation, and Azimuth information, the x, y, and z values of each point cloud in the spatial coordinate system are calculated. Calculate Range * Azimuth to get the x value, calculate Range * Elevation to get the z value, and finally calculate sqrt(Range 2 – x 2 – z 2 ) to get the y value. After obtaining the basic information, ground clutter, traversable targets, and point cloud information with low signal-to-noise ratio are filtered out.

[0109] Step S2, dynamic and static point cloud segmentation. The specific implementation method is as follows:

[0110] According to the characteristic that the number of static point clouds in the 4D millimeter-wave radar point cloud is usually more than that of dynamic point clouds under normal vehicle driving conditions, the Doppler velocity information is compensated according to the azimuth information of the point cloud, and the velocity boundary value for segmenting the dynamic point cloud and the static point cloud is calculated and obtained (equivalent to obtaining the Doppler velocity threshold of at least one spatial point mentioned above), and the point cloud is divided into a static point cloud and a dynamic point cloud (equivalent to determining the spatial points with Doppler velocity less than the Doppler velocity threshold as static points; determining the spatial points with Doppler velocity greater than or equal to the Doppler velocity threshold as dynamic points).

[0111] Step S3, perform multi-dimensional wavelet clustering on the dynamic and static point clouds respectively, and the specific implementation method is as follows:

[0112] Map the dynamic and static point cloud data to the spatial coordinate grid matrix, Doppler velocity grid matrix, and RCS grid matrix respectively according to the preset grid division parameters (equivalent to mapping at least one spatial point to the preset grid system associated with the vehicle body coordinate system mentioned above to establish the spatial coordinate grid matrix, Doppler velocity grid matrix, and radar cross-section grid matrix), fill the divided spatial coordinate grid matrix, perform convolution operation using the selected wavelet basis Wavelet, perform filtering calculation on the convolved spatial coordinate grid matrix through the set threshold parameter Threshold-Wave to obtain the connected domain matrix, perform dilation calculation on the Doppler velocity grid matrix and the RCS grid matrix, and perform weighted intersection with the connected domain matrix to update the connected domain matrix, and then perform the connected domain search calculation, assign a cluster ID ClusterID to each grid cluster, form a new ClusterID matrix, and finally look up the table for each point cloud to obtain its corresponding ClusterID (equivalent to processing the spatial coordinate grid matrix, Doppler velocity grid matrix, and radar cross-section grid matrix mentioned above to determine the cluster identification numbers of each spatial point).

[0113] Step S4, perform association operation based on the wavelet clustering grid to filter out noise targets, and the specific implementation method is as follows:

[0114] Calculate the intersection of the connected domain matrices of the point cloud data of the current frame and the previous frame to obtain the intersection matrix, traverse each cluster of the current frame, divide the number of grids in the intersection matrix within the grid range corresponding to each cluster by the number of grids of each cluster, and when the ratio is greater than or equal to the threshold parameter Threshold-Association, determine that these two clusters are associated clusters, and when the ratio is less than the threshold parameter Threshold-Association, determine that these two clusters are not associated clusters. (Equivalent to calculating the intersection matrix according to the connected domain matrix of the i-th frame and the connected domain matrix of the (i - 1)-th frame, and by comparing the association ratio with the preset threshold, the association relationship between the clustering clusters in the i-th frame and the corresponding clustering clusters in the (i - 1)-th frame can be judged)

[0115] In the process of traditional wavelet clustering calculation, if the dimension of the input data exceeds 2D, the common method is to reduce the dimension of the multi-dimensional data to 2D for calculation, usually achieved by directly expanding and splicing the multi-dimensional matrix. This means that for each additional clustering dimension of the input information, the time cost and memory requirement of the calculation process will increase significantly. To solve the above problems, the embodiments of the present application propose an efficient wavelet clustering method for multi-dimensional input data, and different clustering strategies are designed for dynamic and static point clouds.

[0116] Combine Figure 3 Describe the implementation method of multi-dimensional wavelet clustering of dynamic point clouds in step S3 above, including the following steps:

[0117] Step S3.1, establish a grid matrix according to the x, y, z, Doppler, and RCS information of the dynamic point cloud. The specific implementation method is as follows:

[0118] Set the grid number parameter GridNum in the x, y, and z directions, determine the value of the GridNum parameter according to the corresponding resolution parameter of the used 4D millimeter-wave radar, and obtain the grid side length parameter GridSide in the x, y, and z directions by dividing the detection distance value of the radar in each direction by GridNum. First, map each point cloud to the spatial coordinate grid matrix according to the magnitudes of the x, y, z, and GridSide values. The value of each element in the matrix is the total number of point clouds falling within the grid, PointNum. At the same time, establish two matrices of the same size as the spatial coordinate grid matrix, calculate and fill the Doppler velocity mean and RCS mean into the elements of the matrices respectively, and generate the Doppler velocity grid matrix and the RCS grid matrix. (Equivalent to mapping at least one spatial point to the preset grid system associated with the vehicle body coordinate system according to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the point cloud information above, and establishing the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross-section grid matrix)

[0119] Step S3.2, perform dilation calculation on the Doppler velocity grid matrix and the RCS grid matrix. The specific implementation method is as follows:

[0120] Traverse the Doppler velocity grid matrix and the RCS grid matrix obtained in step S3.1, and inflate the non-zero values in the two matrices with a window of size 3*3. Different from the conventional image inflation calculation, each non-zero grid will be subjected to the inflation operation, and the grids in the 8 adjacent directions of this grid will be filled with the same Doppler velocity or RCS value as this grid. (Equivalent to the above-mentioned inflation processing of the Doppler velocity grid matrix and the radar cross-section grid matrix to determine the Doppler velocity and radar cross-section of the associated grids of each target grid in the Doppler velocity grid matrix and the radar cross-section grid matrix, obtaining the inflated Doppler velocity grid matrix and the inflated radar cross-section grid matrix, where the target grid is the grid corresponding to the non-zero value in the Doppler velocity grid matrix or the radar cross-section grid matrix)

[0121] Step S3.3, perform wavelet transform on the spatial coordinate grid matrix using the set wavelet basis. The specific implementation method is as follows:

[0122] Apply wavelet transform to the spatial coordinate grid matrix obtained in step S3.1. The actual calculation process is to perform convolution calculation after filling the spatial coordinate grid matrix, and perform one-dimensional column convolution and one-dimensional row convolution in sequence according to the used wavelet basis function, or directly perform two-dimensional convolution. (Equivalent to the above-mentioned wavelet transform of the spatial coordinate grid matrix to obtain the wavelet matrix)

[0123] Step S3.4, filter the convolved matrix using the set threshold parameter Threshold-Wave. The specific implementation method is as follows:

[0124] Filter the matrix obtained by the convolution calculation in step S3.3 according to the set threshold parameter Threshold-Wave, set the grid elements smaller than the threshold parameter Threshold-Wave to 0, and keep the grid elements greater than or equal to the threshold parameter Threshold-Wave unchanged. (Equivalent to the above-mentioned setting the values of the elements in the wavelet matrix smaller than the preset wave threshold to zero values to filter and obtain the filtered matrix)

[0125] Step S3.5, find the connected regions according to the spatial coordinate matrix, Doppler velocity matrix, and RCS matrix and assign ClusterID to the matrix elements. The specific implementation method is as follows:

[0126] Traverse the filtered grid coordinate matrix obtained in step S3.4. When the traversed element is non-zero, search for the elements within the 8 adjacent directions of this element. If this element is also non-zero, then make a judgment. When the difference between the two elements in the Doppler grid matrix corresponding to the positions of these two elements is less than the threshold parameter Threshold-Doppler, and the difference between the two elements in the RCS grid matrix corresponding to the positions of these two elements is less than the threshold parameter Threshold-RCS, only then assign the same ClusterID value to the searched element as this element; otherwise, consider the searched element and this element to be different clusters and do not assign them the same ClusterID value. When all the elements in the grid coordinate matrix have been traversed, end the search for connected component calculation. (This is equivalent to, in the above case where the Doppler velocity difference between the target element and its associated elements in the dilated Doppler velocity grid matrix corresponding to the filtered matrix is less than the preset Doppler velocity threshold, and the radar cross-section difference between the target element and its associated elements in the dilated radar cross-section grid matrix is less than the preset radar cross-section threshold, determining the target element and its associated elements as the same clustering cluster to obtain the connected component matrix, where the target element is any non-zero numerical element in the filtered matrix, and the associated element is a non-zero numerical element in the filtered matrix that is connected to the target element)

[0127] Step S3.6, map the cluster class label of each point cloud back to the initial feature grid. The specific implementation method is as follows:

[0128] Calculate the grid element index where the point cloud is located according to the x, y, z values of each point cloud in the current frame and the GridSide parameter, search for the cluster identification number within the corresponding element of the ClusterID matrix calculated in step S3.5, and obtain the cluster identification number of each point cloud. (This is equivalent to, in the above case, assigning cluster identification numbers to each clustering cluster in the connected component matrix to obtain the label matrix, where different clustering clusters correspond to different cluster identification numbers; mapping the label matrix to the preset grid system to determine the cluster identification numbers of each spatial point). Among them, the point cloud with a ClusterID value of 0 is regarded as an isolated noise point and filtered out. Since the mean calculation is introduced in the calculation process of the Doppler velocity matrix and the RCS matrix, the adverse effects of a small number of noise points in the point cloud data on the clustering performance are effectively avoided.

[0129] The calculation process of the multi-dimensional wavelet clustering method for static target point clouds is as Figure 4As shown, it has little difference from the multi-dimensional wavelet clustering calculation process of the dynamic point cloud. The main difference is that compared with the dynamic point cloud, the Doppler velocities of the static point cloud are all near 0, and it is impossible to distinguish dynamic targets from static targets by the difference in Doppler velocity values. Therefore, only the RCS matrix is used to judge whether the grid elements are connected. Since the RCS of different types of targets is different, different types of targets that are close to each other in physical space, such as a vehicle and a person beside the vehicle, can be distinguished by this method.

[0130] Combined with Figure 5 The implementation method of the fast association method based on the wavelet clustering grid in step S4 is described as follows:

[0131] Step S4.1: Calculate the intersection of the connected component matrices of the point cloud data of the current frame and the previous frame. The specific implementation method is as follows:

[0132] Calculate the intersection of the connected component matrices of the point cloud data of the current frame and the previous frame to obtain the intersection matrix. When the elements in both matrices are not 0, the calculation result of the corresponding element in the intersection matrix is 1; otherwise, the result of the element in the intersection matrix is assigned 0. (This is equivalent to calculating the intersection matrix according to the connected component matrix of the i-th frame and the connected component matrix of the (i - 1)-th frame. The intersection matrix includes the first value and the second value. The first value indicates that any element in the connected component matrix of the i-th frame and the corresponding element in the connected component matrix of the (i - 1)-th frame are both not 0, and the second value indicates that at least one of any element in the connected component matrix of the i-th frame and the corresponding element in the connected component matrix of the (i - 1)-th frame is 0)

[0133] Step S4.2: Calculate the association ratio between the intersection matrix and each cluster of the current frame. The specific implementation method is as follows:

[0134] Traverse each cluster in the cluster ID matrix of the current frame clustering, count the number of grids occupied by each cluster ClusterNum, and count the number of grids in the intersection matrix calculated in step S4.1 within the range of each cluster AssociationNum. Divide the AssociationNum of each cluster by ClusterNum to obtain the association ratio of the corresponding cluster. (This is equivalent to calculating the association ratio of each clustering cluster in the label matrix of the i-th frame according to the intersection matrix. Each association ratio is the ratio of the number of grids of each clustering cluster in the intersection matrix to the number of grids of each clustering cluster in the label matrix)

[0135] Step S4.3: Judge whether each cluster of the current frame is associated with the corresponding cluster of the previous frame according to the association ratio. The specific implementation method is as follows:

[0136] Traverse each cluster in the ClusterID matrix and make a judgment based on the association ratio calculated in step S4.2. When the ratio is greater than or equal to the threshold parameter Threshold-Association, these two clusters are determined to be associated clusters; when the ratio is less than the threshold parameter Threshold-Association, these two clusters are determined not to be associated clusters. (This is equivalent to determining that the clustering clusters in the i-th frame and the corresponding clustering clusters in the (i - 1)-th frame are associated clusters when the association ratio is greater than or equal to the preset threshold; and determining that the clustering clusters in the i-th frame and the corresponding clustering clusters in the (i - 1)-th frame are not associated clusters when the association ratio is less than the preset threshold)

[0137] Figure 6 It is a schematic diagram of the clustering effect of the multi-dimensional wavelet clustering algorithm. The legend is the point cloud information from the top-down perspective of the Bird's Eye View (BEV). In the figure, the blue point cloud is the static point cloud, the red point cloud is the dynamic point cloud, the green box is the Box fitted and marked according to the ClusterID of the point cloud, the yellow area is outside the Field of View (FOV) edge, and the dotted line is the virtual lane line. The left figure is the clustering result obtained by clustering only using the spatial coordinate information x, y, z, and the right figure is the result of adding Doppler information and RCS information to participate in the clustering. It can be seen from the part marked by the circle in the figure that after using multi-dimensional clustering, different targets that are extremely close in physical space are effectively divided, and the clustering performance is significantly improved.

[0138] The present invention proposes an efficient multi-dimensional wavelet clustering method for 4D millimeter-wave radar point clouds and a fast association method based on wavelet clustering grids. Compared with traditional density-based clustering algorithms such as DBSCAN, the calculation time-consuming in the clustering calculation process is greatly reduced, and Doppler velocity information and RCS information are introduced to participate in the clustering. At the same time, by calculating the average Doppler velocity and the average RCS in the grid, the negative impact of noise points on the clustering result is reduced to improve the robustness of the algorithm. Compared with the traditional wavelet clustering method, a new Doppler velocity grid matrix and RCS grid matrix are also used to accelerate the calculation process and reduce the memory requirement. The algorithm proposed by the present invention can effectively distinguish the same category or different category targets with different Doppler velocities or different radar cross-sections (RCS) that are close in physical space, effectively improving the performance of the clustering algorithm, and is not affected by the number of points, and can achieve fast and high-precision clustering processing of a large number of point data.

[0139] Based on the point cloud clustering method provided in the above embodiments, correspondingly, the present application also provides a specific implementation manner of the point cloud clustering device. Please refer to the following embodiments.

[0140] Please refer to Figure 7, a point cloud clustering device 700 provided by an embodiment of the present application may include the following modules: a first acquisition module 701, a construction module 702, a first determination module 703, and a clustering module 704.

[0141] The first acquisition module 701 is configured to acquire point cloud information detected by a vehicle at the i-th frame. The point cloud information includes point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross section.

[0142] The construction module 702 is configured to map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the coordinate information, Doppler velocity, and radar cross section of at least one spatial point in the point cloud information, and construct a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross section grid matrix. The preset grid system includes a plurality of grids. The spatial coordinate grid matrix includes values of a plurality of first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes values of a plurality of second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross section grid matrix includes values of a plurality of third elements, and the value of each third element is the average radar cross section of the spatial points in the corresponding grid.

[0143] The first determination module 703 is configured to process the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross section grid matrix to determine the cluster identification numbers of each spatial point.

[0144] The clustering module 704 is configured to cluster at least one spatial point into at least one point cloud cluster according to the cluster identification numbers of each spatial point, and the cluster identification numbers of the spatial points in different point cloud clusters are different.

[0145] The point cloud clustering device according to the embodiment of the present application can map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the point cloud information detected by the vehicle in the i-th frame, establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix. The point cloud information includes the point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross-section. The preset grid system includes multiple grids. The spatial coordinate grid matrix includes the values of multiple first elements, and the value of each first element is the number of spatial points in the corresponding grid. The Doppler velocity grid matrix includes the values of multiple second elements, and the value of each second element is the average Doppler velocity of the spatial points in the corresponding grid. The radar cross-section grid matrix includes the values of multiple third elements, and the value of each third element is the average radar cross-section of the spatial points in the corresponding grid. Process the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross-section grid matrix to determine the cluster identification number of each spatial point. According to the cluster identification number of each spatial point, cluster at least one spatial point into at least one point cloud cluster, and the cluster identification numbers of the spatial points in different point cloud clusters are different. In this way, in the embodiment of the present application, when performing point cloud clustering according to the coordinate information of at least one spatial point, the Doppler velocity and radar cross-section of at least one spatial point are also introduced to participate in the clustering. In this way, through the average Doppler velocity in the Doppler velocity grid matrix and the average radar cross-section in the radar cross-section grid matrix, the negative impact of noise points on the clustering result can be reduced, thereby improving the algorithm robustness, effectively distinguishing different targets close in physical space, and improving the accuracy of point cloud clustering.

[0146] In some embodiments, the above first determination module 703 may specifically include:

[0147] An inflation unit for performing inflation processing on the Doppler velocity grid matrix and the radar cross-section grid matrix to determine the Doppler velocity and radar cross-section of the associated grids of each target grid in the Doppler velocity grid matrix and the radar cross-section grid matrix, and obtaining an inflated Doppler velocity grid matrix and an inflated radar cross-section grid matrix. The target grid is the grid corresponding to the non-zero value in the Doppler velocity grid matrix or the radar cross-section grid matrix;

[0148] A transformation unit for performing wavelet transformation on the spatial coordinate grid matrix to obtain a wavelet matrix;

[0149] A filtering unit for setting the values of the elements in the wavelet matrix that are less than the preset wave threshold to zero values, and filtering to obtain a filtered matrix;

[0150] A determination unit, configured to determine, when a Doppler velocity difference between a target element and its associated elements in a filtering matrix corresponding to an inflated Doppler velocity grid matrix is less than a preset Doppler velocity threshold, and a radar cross-section difference between the target element and its associated elements in an inflated radar cross-section grid matrix is less than a preset radar cross-section threshold, the target element and its associated elements as the same clustering cluster, so as to obtain a connected domain matrix, where the target element is any non-zero numerical element in the filtering matrix, and the associated elements are non-zero numerical elements that are connected to the target element in the filtering matrix;

[0151] A labeling unit, configured to assign a cluster identification number to each clustering cluster in the connected domain matrix, so as to obtain a labeling matrix, where different clustering clusters correspond to different cluster identification numbers;

[0152] A mapping unit, configured to map the labeling matrix to a preset grid system to determine the cluster identification numbers of each spatial point.

[0153] As an implementation manner of the present application, in order to effectively verify different targets, the above device 700 may further include:

[0154] A first calculation module, configured to calculate an intersection matrix according to the connected domain matrix of the i-th frame and the connected domain matrix of the (i - 1)-th frame, where the intersection matrix includes a first value and a second value, the first value indicates that any element in the connected domain matrix of the i-th frame and the corresponding element in the connected domain matrix of the (i - 1)-th frame are both non-zero, and the second value indicates that at least one of any element in the connected domain matrix of the i-th frame and the corresponding element in the connected domain matrix of the (i - 1)-th frame is zero;

[0155] A second calculation module, configured to calculate an association ratio of each clustering cluster in the labeling matrix of the i-th frame according to the intersection matrix, where each association ratio is the ratio of the number of grids of each clustering cluster in the intersection matrix to the number of grids of each clustering cluster in the labeling matrix;

[0156] A second determination module, configured to determine that the clustering cluster in the i-th frame and the corresponding clustering cluster in the (i - 1)-th frame are associated clusters when the association ratio is greater than or equal to a preset threshold;

[0157] The second determination module is further configured to determine that the clustering cluster in the i-th frame and the corresponding clustering cluster in the (i - 1)-th frame are not associated clusters when the association ratio is less than the preset threshold.

[0158] As another implementation manner of the present application, in order to effectively distinguish different moving and static targets that are physically close to each other, the above device 700 may further include:

[0159] A second acquisition module, configured to acquire a Doppler velocity threshold of at least one spatial point;

[0160] A third determination module, configured to determine a spatial point with a Doppler velocity less than the Doppler velocity threshold as a static point;

[0161] The third determination module is further configured to determine a spatial point with a Doppler velocity greater than or equal to a Doppler velocity threshold as a dynamic point;

[0162] The above-mentioned establishment module 702 may specifically include:

[0163] The first establishment unit is configured to map at least one dynamic point to a preset grid system associated with the vehicle body coordinate system according to the coordinate information, Doppler velocity, and radar cross-section of at least one dynamic point, and establish a first subspace coordinate grid matrix, a Doppler velocity grid matrix, and a first sub-radar cross-section grid matrix;

[0164] The second establishment unit is configured to map at least one static point to a preset grid system according to the coordinate information and radar cross-section of at least one static point, and establish a second subspace coordinate grid matrix and a second sub-radar cross-section grid matrix;

[0165] The spatial coordinate grid matrix includes a first subspace coordinate grid matrix and a second subspace coordinate grid matrix, and the radar cross-section grid matrix includes a first sub-radar cross-section grid matrix and a second sub-radar cross-section grid matrix.

[0166] In some embodiments, the above-mentioned device 700 may further include:

[0167] The first filtering module is configured to filter out point information with a radar cross-section lower than a preset signal-to-noise ratio threshold in the point cloud information to obtain first point cloud information;

[0168] The above-mentioned establishment module 702 is further configured to map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the first point cloud information, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix.

[0169] In some embodiments, the above-mentioned device 700 may further include:

[0170] The third acquisition module is configured to acquire the ground clutter range of the vehicle in the i-th frame;

[0171] The second filtering module is configured to filter out point information indicating that the spatial point is within the ground clutter range in the point cloud information to obtain second point cloud information;

[0172] The above-mentioned establishment module 702 is further configured to map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the second point cloud information, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix.

[0173] In some embodiments, the above-mentioned device 700 may further include:

[0174] A third filtering module, configured to filter out point information indicating that the spatial points in the point cloud information are outside a preset height threshold range, so as to obtain third point cloud information;

[0175] The above-mentioned establishing module 702 is further configured to map at least one spatial point to a preset grid system associated with the vehicle body coordinate system according to the coordinate information, Doppler velocity, and radar cross-section of at least one spatial point in the third point cloud information, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross-section grid matrix.

[0176] Figure 8 FIG. shows a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application.

[0177] The electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0178] Specifically, the above-mentioned processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0179] The memory 802 may include a mass storage for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In a suitable case, the memory 802 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 802 may be internal or external to the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory.

[0180] In a specific embodiment, the memory 802 may include a read-only memory (ROM), a random access memory (RAM), a disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in reference to the method according to one aspect of the present disclosure.

[0181] The processor 801 reads and executes the computer program instructions stored in the memory 802 to implement any one of the point cloud clustering methods in the above embodiments.

[0182] In one example, the electronic device may further include a communication interface 803 and a bus 810. Among them, as Figure 8 shown, the processor 801, the memory 802, and the communication interface 803 are connected through the bus 810 and complete communication with each other.

[0183] The communication interface 803 is mainly used to implement communication between each module, device, unit, and / or device in the embodiments of the present application.

[0184] The bus 810 includes hardware, software, or both, and couples the components of the electronic device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, the bus 810 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0185] The electronic device can execute the point cloud clustering method in the embodiments of the present application, thereby implementing the point cloud clustering method and device described in combination with Figure 1 and Figure 7 described.

[0186] In addition, in combination with the point cloud clustering method in the above embodiments, the embodiments of the present application can be implemented by providing a computer-readable storage medium. Computer program instructions are stored on the computer-readable storage medium; when the computer program instructions are executed by a processor, any one of the point cloud clustering methods in the above embodiments is implemented.

[0187] In combination with the point cloud clustering method in the above embodiments, the embodiments of the present application can provide a computer program product. When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device executes the point cloud clustering method as described above in any item.

[0188] Combined with the point cloud clustering method in the above embodiments, an embodiment of the present application can provide a vehicle to implement it. The vehicle includes at least one of the following: the point cloud clustering device as described above; the electronic device as described above; the computer-readable storage medium as described above; the computer program product as described above.

[0189] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0190] The functional blocks shown in the above structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave on a transmission medium or a communication link. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, intranet, and so on.

[0191] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0192] As described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowchart and / or block diagram, and the combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It should also be understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0193] As described above, the foregoing is only a specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present application.

Claims

1. A point cloud clustering method, characterized in that: include: Acquire point cloud information detected by the vehicle in the i-th frame, wherein the point cloud information includes point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity, and radar cross section; According to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the point cloud information, the at least one spatial point is mapped to a preset grid system associated with the body coordinate system of the vehicle, and a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix are established, wherein the preset grid system includes a plurality of grids, the spatial coordinate grid matrix includes a plurality of first element values, each of which is the number of spatial points in the corresponding grid, the Doppler velocity grid matrix includes a plurality of second element values, each of which is the average Doppler velocity of the spatial points in the corresponding grid, and the radar cross section grid matrix includes a plurality of third element values, each of which is the average radar cross section of the spatial points in the corresponding grid; Processing the spatial coordinate grid matrix, the Doppler velocity grid matrix and the radar cross section grid matrix to determine a cluster identification number for each of the spatial points; According to the cluster identification number of each of the spatial points, the at least one spatial point is clustered into at least one point cloud cluster, and the cluster identification numbers of the spatial points in different point cloud clusters are different.

2. The method according to claim 1, characterized in that The processing of the spatial coordinate grid matrix, the Doppler velocity grid matrix, and the radar cross section grid matrix to determine the cluster identification number of each of the spatial points includes: The Doppler velocity grid matrix and the radar cross section grid matrix are expanded to determine the Doppler velocity and radar cross section of the associated grid of each target grid in the Doppler velocity grid matrix and the radar cross section grid matrix, and obtain an expanded Doppler velocity grid matrix and an expanded radar cross section grid matrix, wherein the target grid is a grid corresponding to a non-zero value in the Doppler velocity grid matrix or the radar cross section grid matrix; Performing wavelet transform on the space coordinate grid matrix to obtain a wavelet matrix; The values ​​of the elements in the wavelet matrix that are smaller than a preset wavelet threshold are set to zero, and filtering is performed to obtain a filter matrix; In the case where the Doppler velocity difference between the target element and its associated element in the expanded Doppler velocity grid matrix in the filter matrix is ​​less than a preset Doppler velocity threshold, and the radar cross section difference between the target element and its associated element in the expanded radar cross section grid matrix is ​​less than a preset radar cross section threshold, the target element and its associated element are determined as the same cluster to obtain a connected domain matrix, wherein the target element is any non-zero value element in the filter matrix, and the associated element is an element in the filter matrix that is connected to the target element and has a non-zero value; Assigning a cluster identification number to each cluster in the connected domain matrix to obtain a label matrix, wherein different clusters correspond to different cluster identification numbers; The label matrix is ​​mapped to the preset grid system to determine the cluster identification number of each of the spatial points.

3. The method according to claim 2, characterized in that After mapping the label matrix to the preset grid system and determining the cluster identification number of each of the spatial points, the method further includes: An intersection matrix is ​​calculated according to the connected domain matrix of the i-th frame and the connected domain matrix of the i-1-th frame, wherein the intersection matrix includes a first value and a second value, wherein the first value indicates that any element in the connected domain matrix of the i-th frame and the corresponding element in the connected domain matrix of the i-1-th frame are not 0, and the second value indicates that at least one of the elements in the connected domain matrix of the i-th frame and the corresponding element in the connected domain matrix of the i-1-th frame is 0; According to the intersection matrix, calculating the correlation ratio of each cluster in the label matrix of the i-th frame, each correlation ratio is the ratio of the number of grids of each cluster in the intersection matrix to the number of grids of each cluster in the label matrix; When the correlation ratio is greater than or equal to a preset threshold, determining that the cluster in the i-th frame and the corresponding cluster in the i-1-th frame are associated clusters; When the association ratio is less than the preset threshold, it is determined that the cluster in the i-th frame and the corresponding cluster in the i-1-th frame are not the associated clusters.

4. The method according to claim 1, characterized in that: Before mapping the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of the at least one spatial point in the point cloud information, and establishing a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, the method further includes: Acquire a Doppler velocity threshold of the at least one spatial point; Determine the spatial point whose Doppler velocity is less than the Doppler velocity threshold as a static point; Determine a spatial point whose Doppler velocity is greater than or equal to the Doppler velocity threshold as a dynamic point; The method maps the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the point cloud information, and establishes a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, including: According to the coordinate information, Doppler velocity and radar cross section of at least one of the dynamic points, the at least one dynamic point is mapped to a preset grid system associated with the body coordinate system of the vehicle, and a first subspace coordinate grid matrix, a Doppler velocity grid matrix and a first sub-radar cross section grid matrix are established; According to the coordinate information and radar cross section of at least one of the static points, mapping the at least one static point to the preset grid system, and establishing a second subspace coordinate grid matrix and a second sub-radar cross section grid matrix; The space coordinate grid matrix includes the first sub-space coordinate grid matrix and the second sub-space coordinate grid matrix, and the radar cross section grid matrix includes the first sub-radar cross section grid matrix and the second sub-radar cross section grid matrix.

5. The method according to claim 1, characterized in that: Before mapping the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of the at least one spatial point in the point cloud information, and establishing a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, the method further includes: In the point cloud information, point information whose radar cross section is lower than a preset signal-to-noise ratio threshold is filtered out to obtain first point cloud information; The method maps the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the point cloud information, and establishes a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, including: According to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the first point cloud information, the at least one spatial point is mapped to a preset grid system associated with the body coordinate system of the vehicle, and a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix are established.

6. The method according to claim 1, characterized in that Before mapping the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of the at least one spatial point in the point cloud information, and establishing a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, the method further includes: Obtaining the ground clutter range of the vehicle in the i-th frame; In the point cloud information, point information whose coordinate information indicates that the spatial point is within the ground clutter range is filtered out to obtain second point cloud information; The method maps the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the point cloud information, and establishes a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, including: According to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the second point cloud information, the at least one spatial point is mapped to a preset grid system associated with the body coordinate system of the vehicle, and a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix are established.

7. The method according to claim 1, characterized in that Before mapping the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of the at least one spatial point in the point cloud information, and establishing a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, the method further includes: In the point cloud information, point information whose coordinate information indicates that the spatial point is outside a preset height threshold range is filtered out to obtain third point cloud information; The method maps the at least one spatial point to a preset grid system associated with the body coordinate system of the vehicle according to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the point cloud information, and establishes a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix, including: According to the coordinate information, Doppler velocity and radar cross section of at least one spatial point in the third point cloud information, the at least one spatial point is mapped to a preset grid system associated with the body coordinate system of the vehicle, and a spatial coordinate grid matrix, a Doppler velocity grid matrix and a radar cross section grid matrix are established.

8. A point cloud clustering device, characterized in that: The device comprises: A first acquisition module is used to acquire point cloud information detected by the vehicle in the i-th frame, wherein the point cloud information includes point information of at least one spatial point, and the point information includes coordinate information, Doppler velocity and radar cross section; an establishing module, configured to map the at least one spatial point in the point cloud information to a preset grid system associated with the vehicle body coordinate system, and establish a spatial coordinate grid matrix, a Doppler velocity grid matrix, and a radar cross section grid matrix, according to the coordinate information, Doppler velocity, and radar cross section of the at least one spatial point in the point cloud information, wherein the preset grid system comprises a plurality of grids, the spatial coordinate grid matrix comprises a plurality of first element values, each of which is the number of spatial points in the corresponding grid, the Doppler velocity grid matrix comprises a plurality of second element values, each of which is the average Doppler velocity of the spatial points in the corresponding grid, and the radar cross section grid matrix comprises a plurality of third element values, each of which is the average radar cross section of the spatial points in the corresponding grid; A first determination module is used to process the spatial coordinate grid matrix, the Doppler velocity grid matrix and the radar cross section grid matrix to determine the cluster identification number of each of the spatial points; The clustering module is used to cluster the at least one spatial point into at least one point cloud cluster according to the cluster identification number of each spatial point, and the cluster identification numbers of the spatial points in different point cloud clusters are different.

9. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the point cloud clustering method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the point cloud clustering method according to any one of claims 1 to 7 is implemented.

Images