Millimeter wave radar point cloud data processing method and device

Abstract

The invention discloses a millimeter wave radar point cloud data processing method and device. The method is applied to an FPGA (Field Programmable Gate Array), and comprises the following steps: sequentially receiving multiple pieces of point cloud data contained in a point cloud data frame; performing Doppler compensation processing and clutter labeling processing on the multiple pieces of point cloud data in parallel to obtain multiple pieces of processed point cloud data; when receiving of the point cloud data frame is completed, determining a target dynamic and static point cloud discrimination threshold value corresponding to the point cloud data frame according to the compensated point cloud data; performing dynamic and static point cloud labeling and gridding processing on the plurality of processed point cloud data in parallel through clutter identifiers and dynamic and static point cloud discrimination threshold values to obtain gridded point cloud data frames; wavelet clustering is carried out on the point cloud data frames after gridding processing, and a cluster to which each piece of point cloud data belongs in the point cloud data frames is determined. According to the scheme provided by the invention, the data processing complexity of the millimeter wave radar system can be reduced, and the processing efficiency of the point cloud data is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and device for processing millimeter-wave radar point cloud data. Background Art

[0002] Millimeter-wave radar is an important sensor device in the field of autonomous driving. Target tracking can be achieved by analyzing the point cloud data detected by the millimeter-wave radar. Usually, clustering algorithms are used to analyze point cloud data, and point clouds belonging to the same target are divided into a cluster. Then, based on the clusters, association and tracking can be performed to determine the distance, speed and other information of the target.

[0003] In related technologies, the digital signal of the radar is usually processed by FPGA (Field Programmable Gate Array) to generate point cloud data, and then the point cloud data is clustered by CPU. This method not only increases the hardware complexity of the millimeter wave radar system, but also increases the complexity of software development. Moreover, when processing point cloud data, it is usually processed in a sequential manner, that is, after completing the processing of one frame of point cloud data, the next frame of point cloud data will be processed. This method increases the time complexity of point cloud processing and reduces the efficiency of point cloud data processing.

[0004] It can be seen that in the relevant technology, the processing of point cloud data has the problems of high complexity and low efficiency. Summary of the invention

[0005] The embodiments of the present application provide a method and device for processing millimeter-wave radar point cloud data, which can reduce the data processing complexity of the millimeter-wave radar system and improve the processing efficiency of point cloud data.

[0006] In a first aspect, an embodiment of the present application provides a method for processing millimeter-wave radar point cloud data, which is applied to a field programmable gate array FPGA, and the method includes: sequentially receiving multiple point cloud data contained in a point cloud data frame; performing Doppler compensation processing and clutter labeling processing on the multiple point cloud data in parallel to obtain multiple processed point cloud data; when the point cloud data frame is received, determining the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame according to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold; performing dynamic and static point cloud labeling and gridding processing on the multiple processed point cloud data in parallel through the clutter identifier and the dynamic and static point cloud discrimination threshold to obtain a gridded point cloud data frame, wherein the clutter identifier is used to characterize whether the point cloud data contains ground clutter; performing wavelet clustering on the gridded point cloud data frame to determine the cluster cluster to which each point cloud data in the point cloud data frame belongs, wherein the cluster cluster is used to characterize the target to which the corresponding point cloud data belongs.

[0007] In a second aspect, an embodiment of the present application provides a millimeter-wave radar point cloud data processing device, which is applied to a field programmable gate array FPGA, and the device includes: a point cloud receiving module, which is used to sequentially receive multiple point cloud data contained in a point cloud data frame; a first processing module, which is used to perform Doppler compensation processing and clutter labeling processing on multiple point cloud data in parallel to obtain multiple processed point cloud data; a point cloud discrimination module, which is used to determine the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame according to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold when completing the reception of the point cloud data frame; a second processing module, which is used to perform dynamic and static point cloud labeling and gridding processing on multiple processed point cloud data in parallel through clutter identification and dynamic and static point cloud discrimination threshold to obtain a gridded point cloud data frame, wherein the clutter identification is used to characterize whether the point cloud data contains ground clutter; a clustering module, which is used to perform wavelet clustering on the gridded point cloud data frame to determine the cluster cluster to which each point cloud data in the point cloud data frame belongs, wherein the cluster cluster is used to characterize the target to which the corresponding point cloud data belongs.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for processing millimeter wave radar point cloud data as described in the first aspect is implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the method for processing millimeter-wave radar point cloud data as described in the first aspect is implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method for processing millimeter-wave radar point cloud data as described in the first aspect.

[0011] From the above content, it can be seen that in the embodiment of the present application, the processing and clustering of point cloud data are realized by FPGA. Compared with the processing and clustering of point cloud data by FPGA and CPU, the solution provided by the present application can reduce the hardware complexity and software development complexity of the millimeter wave radar system. In addition, in the embodiment of the present application, in the process of processing point cloud data, Doppler compensation and clutter annotation are performed on the point cloud data in parallel, and the dynamic and static point annotation and gridding of the point cloud data are performed in parallel, which reduces the time complexity of data processing and further improves the processing efficiency of point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 It is a flowchart of a method for processing millimeter wave radar point cloud data provided by an embodiment of the present application;

[0014] Figure 2 This is a schematic diagram of the system structure of a method for processing millimeter wave radar point cloud data provided by an embodiment of the present application;

[0015] Figure 3 is a schematic diagram of a radar detecting target provided by an embodiment of the present application;

[0016] Figure 4 is a histogram for distinguishing dynamic and static point clouds provided by an embodiment of the present application;

[0017] Figure 5 is a schematic diagram of gridded data provided by an embodiment of the present application;

[0018] Figure 6 is a schematic diagram of a Doppler expansion result provided by an embodiment of the present application;

[0019] Figure 7 is a schematic diagram of a wavelet transform result provided by an embodiment of the present application;

[0020] Figure 8 It is a schematic diagram of a pipeline design of grid BRAM reuse and convolution calculation provided by an embodiment of the present application;

[0021] Fig. 9 is a schematic diagram of a connected domain search result provided by an embodiment of the present application;

[0022] Fig.10 is a schematic diagram of the structure of a device for processing millimeter wave radar point cloud data provided by another embodiment of the present application;

[0023] Fig.11 It is a structural schematic diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0024] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0025] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0026] For ease of understanding, before explaining the solution provided in the present application, the background of the solution provided in the present application is first explained.

[0027] Millimeter-wave radar is a key sensor device in the field of autonomous driving. In millimeter-wave radar equipment, the target to be detected is mapped into point cloud data containing the spatial and motion characteristics of the target. In traditional radar point cloud data processing solutions, clustering algorithms are usually used to divide the point clouds belonging to the same target into a cluster, and then association and tracking are performed on the basis of clustering to obtain information such as the distance and speed of the target.

[0028] In related technologies, there are two main methods for point cloud clustering. One is to directly perform clustering analysis on point clouds, such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise); the other is to convert point clouds into grids and then perform grid clustering. The time complexity of DBSCAN is usually O(n 2) or O(nlogn), where n is the number of point clouds. Scenes with a large number of point clouds usually have a higher time complexity, and the processing speed corresponding to different numbers of point clouds may vary greatly. In contrast, the time complexity of grid clustering is usually O(n), where n is the number of grids, and it is insensitive to the order of input data, which is conducive to parallel computing.

[0029] Wavelet clustering is a density-based grid clustering algorithm. The grid matrix and the wavelet basis are transformed to obtain the low-frequency components of the data, thereby showing the clustering properties of the data. The hardware implementation of the wavelet clustering algorithm is usually based on the CPU, while the signal processing of the millimeter-wave radar is usually based on the FPGA. Therefore, the wavelet clustering of point cloud data based on FPGA can not only realize the integration of radar signal processing and data processing algorithms and reduce the complexity of the system, but also the convolution calculation in wavelet clustering can also be realized by FPGA to achieve hardware parallel pipeline acceleration.

[0030] In the conventional signal chain of millimeter-wave radar, signal processing FFT (Fast Fourier Transform) is usually implemented based on FPGA, and point cloud clustering is usually implemented based on CPU. From a system level perspective, a millimeter-wave radar system that includes both FPGA and CPU has high complexity in both hardware design and software development. From the perspective of algorithm implementation, wavelet clustering requires traversal of point cloud data and grid data, and each step of the traversal is reflected on the CPU as several multiplication, addition and other operations. It can be seen that in related technologies, the processing of point cloud data has the problems of high complexity and low efficiency.

[0031] To solve the above problems, the embodiment of the present application provides a method and device for processing millimeter wave radar point cloud data. In the embodiment of the present application, the processing of point cloud data and wavelet clustering are realized on FPGA through pipeline design. This method makes full use of the parallel computing capability of FPGA, realizes point cloud meshing and grid data wavelet transformation through pipeline design, and reduces the complexity of the system while improving the efficiency of clustering algorithm implementation.

[0032] The following first introduces the method for processing millimeter wave radar point cloud data provided by the embodiment of the present application. As mentioned above, the method proposed in the embodiment of the present application is applied in FPGA.

[0033] Figure 1 FIG. 1 is a flow chart showing a method for processing millimeter wave radar point cloud data provided by an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0034] Step S101, receiving a plurality of point cloud data contained in a point cloud data frame in sequence.

[0035] In step S101, the point cloud data includes at least the following data: the three-dimensional spatial coordinates (x, y, z) of the point cloud, the radial distance R of the spherical coordinates, the cosine value of the azimuth angle Sine of pitch angle sinθ, Doppler velocity v dop , as well as the point cloud motion flag MoveID, the clutter flag FilterID, and the cluster number ClusterID. Among them, the mark in the point cloud motion flag is used to indicate whether the point cloud is a static point cloud or a dynamic point cloud; the mark in the clutter flag is used to indicate whether the point cloud contains ground clutter; the mark in the cluster number is used to indicate the cluster to which the point cloud belongs, that is, the target to which the point cloud belongs.

[0036] In step S101, after the radar detects the point cloud data, the point cloud data is sent to the FPGA, so that the FPGA can receive each point cloud data in the order sent by the radar. In addition, in the embodiment of the present application, the number of point cloud data contained in each point cloud data frame is different according to the different radar detection frequencies.

[0037] Step S102 , performing Doppler compensation processing and clutter marking processing on a plurality of point cloud data in parallel to obtain a plurality of processed point cloud data.

[0038] In the embodiment of the present application, a pipeline design method can be used to implement Doppler compensation and clutter annotation of point cloud data. For example, a point cloud data frame includes point cloud data 1 and point cloud data 2. After the FPGA receives point cloud data 1, Doppler compensation is performed on point cloud data 1, and then clutter is annotated on point cloud data 1; while clutter is annotated on point cloud data 1, Doppler compensation is performed on point cloud data 2. Compared with the related art, in which Doppler compensation is performed on all point cloud data in the point cloud data frame and then clutter is annotated on all point cloud data, the processing efficiency of point cloud data can be improved by performing Doppler compensation processing and clutter annotation processing on the point cloud data in parallel.

[0039] Step S103, when the point cloud data frame is received, the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame is determined according to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold.

[0040] In step S103, after receiving the entire frame of point cloud data, the entire frame of point cloud data is subjected to a dynamic and static point cloud discrimination. In the embodiment of the present application, the target dynamic and static point cloud discrimination threshold corresponding to the frame of point cloud data can be determined by statistically analyzing the Doppler velocity of the entire frame of point cloud data, and then the target dynamic and static point cloud discrimination threshold can be used to determine whether the point cloud data in the point cloud data frame is a static point cloud or a dynamic point cloud. For example, the point cloud data between the target dynamic and static point cloud discrimination thresholds is a static point cloud, and vice versa.

[0041] Step S104 , performing dynamic and static point cloud labeling and gridding processing on a plurality of processed point cloud data in parallel through clutter identification and dynamic and static point cloud discrimination thresholds, and obtaining a gridded point cloud data frame.

[0042] In step S104, the clutter identifier is used to characterize whether the point cloud data contains ground clutter. In the embodiment of the present application, whether the point cloud data contains ground clutter can be determined by judging whether the spatial coordinates corresponding to the point cloud data are in the target three-dimensional space. For example, the point cloud data whose spatial coordinates are in the target three-dimensional space does not contain ground clutter, while the point cloud data whose spatial coordinates are outside the target three-dimensional space contains ground clutter.

[0043] In step S104, after the point cloud data is subjected to dynamic and static point cloud discrimination, the point cloud data can be gridded. Similar to step S102, the dynamic and static point cloud discrimination and gridding can also be implemented in parallel to improve the processing efficiency of the point cloud data. In the embodiment of the present application, after the dynamic and static point cloud discrimination of each point cloud data is completed, the point cloud data can be gridded, and the dynamic and static point cloud discrimination of the next point cloud data can be performed at the same time.

[0044] Step S105 , performing wavelet clustering on the gridded point cloud data frame to determine the cluster to which each point cloud data in the point cloud data frame belongs.

[0045] In step S105, after the gridding process is completed for the entire frame of point cloud data, the wavelet clustering process can be performed on the entire frame of point cloud data. In the embodiment of the present application, the wavelet clustering of point cloud data includes Doppler expansion processing, wavelet change processing, and connected domain search processing. After completing the above processing, the clustering cluster to which each point cloud data in the entire frame of point cloud data (i.e., point cloud data frame) belongs can be determined to determine the target to which the corresponding point cloud data belongs.

[0046] Based on the scheme defined in the above steps S101 to S105, it can be known that in the embodiment of the present application, the processing and clustering of point cloud data is realized by FPGA. Compared with the processing and clustering of point cloud data by FPGA and CPU, the scheme provided by the present application can reduce the hardware complexity and software development complexity of the millimeter wave radar system. In addition, in the embodiment of the present application, in the process of processing point cloud data, Doppler compensation and clutter annotation are performed on the point cloud data in parallel, and dynamic and static point annotation and gridding of point cloud data are performed in parallel, which reduces the time complexity of data processing and further improves the processing efficiency of point cloud data.

[0047] Based on the scheme defined in the above steps S101 to S105, it can be seen that the method proposed in the embodiment of the present application mainly includes two stages, namely, the data preprocessing stage and the wavelet clustering stage. In one example, Figure 2 The system structure diagram for implementing the above method is shown. Figure 2 It can be seen that in the embodiment of the present application, the FPGA includes at least a data preprocessing module and a wavelet clustering module, which are used to implement the preprocessing and wavelet clustering of point cloud data respectively. The data preprocessing module is used to implement Doppler compensation, clutter annotation, point cloud storage and dynamic and static point cloud discrimination of point cloud data, and the wavelet clustering module is used to implement grid processing, Doppler expansion, wavelet transform and connected domain search of point cloud data.

[0048] The following combination Figure 2 The specific implementation process of the method provided in the embodiment of the present application is explained.

[0049] First, in the data preprocessing stage, after receiving the point cloud data, the FPGA performs Doppler compensation and clutter labeling processing on the received point cloud data in parallel.

[0050] Specifically, the following steps are executed in a loop until the Doppler compensation processing and clutter annotation processing of all point cloud data in the point cloud data frame are completed to obtain a plurality of processed point cloud data: first, the first point cloud data is subjected to Doppler compensation processing to obtain the compensated first point cloud data; then, the compensated first point cloud data is subjected to clutter annotation processing, and the second point cloud data is subjected to Doppler compensation processing in parallel; then, the compensated second point cloud data is subjected to clutter annotation processing, and the next point cloud data of the second point cloud data is subjected to Doppler compensation processing in parallel.

[0051] In the above embodiment, the first point cloud data is any one of multiple point cloud data, and the second point cloud data is the next point cloud data of the first point cloud data, that is, when the FPGA receives point cloud data, it receives the second point cloud data after receiving the first point cloud data.

[0052] By adopting a parallel execution method with pipeline design to perform Doppler compensation and clutter annotation on point cloud data, the processing efficiency of point cloud data can be improved, thereby reducing the data processing complexity of millimeter wave radar.

[0053] For Doppler compensation, the Doppler velocity and the azimuth cosine value corresponding to the first point cloud data are obtained; the Doppler velocity is compensated by the azimuth cosine value to obtain the forward Doppler velocity corresponding to the first point cloud data.

[0054] In one example, Figure 3 A schematic diagram of radar detecting a target is shown in FIG. Figure 3 As shown, the azimuth of the target detected by the radar is The spherical coordinate radial distance between the radar and the target is R, and the radial velocity detected by the radar, that is, the Doppler velocity, is v dop , v ⊥ is the velocity perpendicular to the Doppler velocity; v is the compensated Doppler velocity, that is, the forward Doppler velocity.

[0055] Depend on Figure 3 It can be seen that the forward Doppler velocity can be expressed by formula (1):

[0056]

[0057] It can be seen from the above content that the Doppler compensation for the first point cloud data is mainly to compensate for the Doppler velocity of the first point cloud data. Therefore, the compensated first point cloud data at least includes the forward Doppler velocity.

[0058] It can be seen from formula (1) that Doppler compensation for point cloud data requires division calculation, and the logic of implementing division calculation in FPGA is relatively complex. Directly using division calculation will increase the data processing complexity of the millimeter wave radar system. Therefore, in order to reduce the complexity of the millimeter wave radar system, in the embodiment of the present application, the division operation can be avoided by table lookup.

[0059] Specifically, the FPGA in the millimeter wave radar system queries the reciprocal value corresponding to the azimuth cosine value from a preset data table, and calculates the product of the Doppler velocity and the reciprocal value of the azimuth cosine value to obtain the forward Doppler velocity.

[0060] In the above embodiment, the preset data table includes the azimuth cosine value, the reciprocal value of the azimuth cosine value, and the corresponding relationship between the azimuth cosine value and the reciprocal value. Among them, the azimuth cosine value in the preset data table can be a range value or a specific value. In the case where the azimuth cosine value in the preset data table is a range value, each azimuth cosine value corresponds to an reciprocal value. When the azimuth cosine value to be queried is within the range value, the reciprocal value is the reciprocal value of the azimuth cosine value to be queried. Among them, the smaller the range of the azimuth cosine value in the preset data table, the more accurate the reciprocal value obtained by the query.

[0061] like Figure 2 As shown, after Doppler compensation is performed on the entire frame of point cloud data, the FPGA performs dynamic and static point cloud discrimination on each point cloud data in the compensated point cloud data frame.

[0062] Specifically, first obtain the detection scene where the point cloud data detected by the millimeter-wave radar is located, and then determine the coefficient and offset that match the detection scene; then, perform Doppler interval mapping on the forward Doppler velocity corresponding to each point cloud data in the point cloud data frame, and obtain the interval index of the forward Doppler velocity in the Doppler statistical distribution table; then determine the lower quantile and upper quantile of the forward Doppler velocity according to the interval index; finally, determine the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame according to the coefficient, offset, upper quantile, and lower quantile.

[0063] In the above embodiment, the Doppler statistical distribution table is a statistical distribution table constructed by a block storage BRAM, and the interval index includes: the read and write address of the forward Doppler velocity in the Doppler statistical distribution table.

[0064] In the above embodiment, most of the point clouds in the radar point cloud are static targets, for example Figure 4 In the histogram of the dynamic and static point cloud discrimination shown in the figure, the maximum value of the forward Doppler v statistical result of the point cloud is hist max The corresponding forward Doppler velocity v MPV , which can be used as an estimated value of the forward Doppler velocity of the static target. That is, in the embodiment of the present application, the velocity value corresponding to the maximum value of the statistical result of the forward Doppler velocity is the forward Doppler velocity of the static target. Figure 4 In, v MPV is the forward Doppler velocity of the static target.

[0065] The definition is divided into probability α satisfying the condition 0<α<0.5, and the corresponding lower and upper quantiles of Doppler velocity are v α and v 1-α The target dynamic and static point cloud discrimination threshold includes an upper threshold and a lower threshold. The lower threshold and the upper threshold of the target dynamic and static point cloud can be determined by formulas (2) and (3):

[0066]

[0067] In formulas (2) and (3), v th_down is the lower threshold; v th_up is the upper threshold; v α is the lower quantile; v 1-α is the upper quantile; a is the coefficient; b is the offset.

[0068] It should be noted that, in the embodiment of the present application, the coefficient a and the offset b can be determined according to the actual detection scene, and the coefficients and offsets corresponding to different detection scenes may be different. In addition, setting the coefficients and offsets can also avoid v 1-α =v α This causes the upper and lower thresholds of the static point cloud to be equal and the length of the static interval to be 0.

[0069] In addition, it should be noted that after determining the target static point cloud discrimination threshold, the FPGA can determine whether the point cloud is a static point cloud or a dynamic point cloud based on the relationship between the forward Doppler velocity of the point cloud and the target static point cloud discrimination threshold. As an example, when the forward Doppler velocity of the point cloud is between the lower threshold and the upper threshold, that is, v th_down ≤v≤v th_up When , the point cloud is determined to be a static point cloud; when the forward Doppler velocity of the point cloud is outside the lower threshold and the upper threshold, that is, v>v th_up or v <v th_down , the point cloud is determined to be a dynamic point cloud.

[0070] In the above embodiment, after Doppler compensation is performed on the point cloud data, Doppler interval mapping may be performed on the forward Doppler velocity corresponding to the point cloud data frame to obtain an interval index of the forward Doppler velocity in the Doppler statistical distribution table.

[0071] Specifically, first, a preset Doppler bit width and a reserved bit width are obtained, and then the forward Doppler velocity is truncated by using the Doppler bit width and the reserved bit width to obtain the truncated forward Doppler velocity; then an offset is added to the truncated forward Doppler velocity by using the reserved bit width to obtain an interval index of the forward Doppler velocity in the Doppler statistical distribution table.

[0072] In one example, the distribution of the compensated Doppler velocity of a frame of point cloud data, that is, the forward Doppler velocity v, is statistically analyzed. Specifically, v is truncated and offset is added to realize Doppler-interval mapping, and a signed fixed-point number v in a certain range is mapped to an interval index DopIdx(v). The interval index of the forward Doppler velocity in the Doppler statistical distribution table can be expressed by formula (4):

[0073]

[0074] In formula (4), v is the forward Doppler velocity; DopIdx(v) is the interval index of the forward Doppler velocity in the Doppler statistical distribution table; W Dop is the Doppler width; W Sec It is the reserved bit width after truncation.

[0075] In the embodiment of the present application, SDP BRAM (Simple Dual Port BRAM), that is, the above-mentioned block storage BRAM, can be used to construct the Doppler statistical distribution table, and the interval index DopIdx(v) is used as the read and write address of the Doppler statistical distribution BRAM. The count value DopCnt(DopIdx) in the corresponding address of the BRAM is read out, incremented by one, and then written back. At the same time, the minimum value of the index DopIdx is recorded. min And the maximum value of DopIdx max For boundary indexes, only [DopIdx min ,DopIdx max ] The count value in the range is not 0.

[0076] It should be noted that the full pipeline design of the method provided in the embodiment of the present application ensures that the Doppler distribution statistics table is constructed after the point cloud data is received.

[0077] Further, after determining the interval index, the FPGA may determine the lower quantile and the upper quantile of the forward Doppler velocity based on the interval index.

[0078] Specifically, firstly, the data length and quantile probability of the point cloud data frame are obtained to calculate the product of the data length and the quantile probability to obtain the target count value; then, the Doppler statistical distribution table is traversed, and the number of point cloud data corresponding to each index interval is accumulated according to the preset traversal order to obtain the accumulated value; when the accumulated value is greater than or equal to the target count value, the forward Doppler velocity corresponding to the point cloud data in the current index interval is determined as the lower quantile, and the upper quantile corresponding to the lower quantile.

[0079] It should be noted that in the embodiments of the present application, Figure 2 As shown in the figure, while performing Doppler compensation on the point cloud data, point cloud counting and point cloud storage are performed in parallel, and the point cloud data is also written into the SDP BRAM of the point cloud storage. When a frame of point cloud data is received, the data length of the current frame point cloud is obtained as L, and then combined with the quantile probability α, the target count value S can be obtained. α =L×α. In the embodiment of the present application, the quantile probability may be a preset value, which may be determined according to the actual application scenario and the actual measurement result.

[0080] Then, from the index DopIdx of the boundary interval min Start traversing and reading the Doppler probability density table BRAM, accumulating the output count value, and when the accumulated value is greater than or equal to S α When the interval index DopIdx α The corresponding forward Doppler velocity is v α , DopIdx can be achieved by adding offset and padding α to v α DopIdx α and v α The calculation of can be expressed by formulas (5) and (6):

[0081]

[0082]

[0083] At this point, the dynamic and static point cloud discrimination of point cloud data is completed.

[0084] like Figure 2 As shown, the point cloud data needs to be stored before the motion and stillness discrimination is performed on the point cloud data, and the point cloud data needs to be clutter-labeled before the point cloud is stored.

[0085] Specifically, firstly, it is detected whether the spatial coordinates corresponding to the first point cloud data are located in the target three-dimensional space; when the spatial coordinates corresponding to the first point cloud data are located in the target three-dimensional space, the clutter identifier corresponding to the first point cloud data is marked as the first identifier; when the spatial coordinates corresponding to the first point cloud data are located outside the target three-dimensional space, the clutter identifier corresponding to the first point cloud data is marked as the second identifier.

[0086] In the above embodiment, the target three-dimensional space is a three-dimensional space that contains point cloud data of a traversable target but does not contain ground clutter. The first identifier is used to indicate that the first point cloud data does not contain ground clutter, and the second identifier is used to indicate that the first point cloud data contains ground clutter.

[0087] As an example, according to the characteristics of the radar point cloud, the target three-dimensional space R that can pass through the target and the ground clutter can be set. filter , the assignment logic of the clutter identifier FilterID of the point cloud can be expressed by formula (7):

[0088]

[0089] In formula (7), 1 is the first identifier, 0 is the second identifier, and x, y, and z represent the spatial coordinates of the point cloud data.

[0090] Further, such as Figure 2As shown, according to v α Calculate the dynamic and static point cloud segmentation threshold v th_up and v th_down , read the point cloud data from the point cloud storage BRAM, and update the MoveID and FilterID of the point cloud according to the dynamic and static classification rules and the filtering rules of traversable targets and ground clutter, and convert the point cloud data (x, y, z, R, v,sinθ,MoveID,FilterID,0) is sent to the clustering module and then written back to the point cloud storage BRAM through the data selector.

[0091] Specifically, the FPGA executes the following steps in a loop until the dynamic and static point annotation and gridding processing of all point cloud data in the multiple processed point cloud data are completed, and the point cloud data frame after gridding processing is obtained: the dynamic and static point cloud annotation of the third point cloud data is performed through the dynamic and static point cloud discrimination threshold to obtain the point cloud dynamic and static identification corresponding to the third point cloud data; the third point cloud data after the dynamic and static point cloud annotation is gridded through a gridding algorithm matched with the clutter identification and the point cloud dynamic and static identification, and the dynamic and static point cloud annotation of the next point cloud of the third point cloud data is performed in parallel.

[0092] In the above embodiment, the third point cloud data is any one of a plurality of processed point cloud data.

[0093] For example, Figure 2 As shown in the figure, the point cloud after Doppler compensation and traversable target and ground clutter filtering is stored in the point cloud storage module. After a frame of point cloud is received, the dynamic and static point cloud discrimination threshold is calculated, the point cloud is read from the point cloud storage module and the MoveID is updated according to the dynamic and static discrimination threshold. The dynamic point cloud MoveID is assigned to 1 and the static point cloud is assigned to 0. The point cloud after the MoveID update is sent to the wavelet clustering module and is stored again in the point cloud storage module.

[0094] The wavelet clustering module receives the preprocessed point cloud and grids the points where the FilterID is not 0, as shown in the following figure. Figure 5 The gridded data is shown. Set the grid size (X grid_size ,Y grid_size ), the gridding range x∈[x base ,x max ), y∈[y base ,y max ). The mapping relationship from coordinates (x, y) to grid index (i, j) is shown in formula (8):

[0095]

[0096] In formula (8), Indicates rounding down, i,

[0097] Record the number of point clouds mapped to each grid to get the grid count value G m×n ,in, (0,0)≤(i,j)<(m,n). Count the point clouds in each grid and calculate the Doppler velocity accumulation value of the point clouds in the grid.

[0098] In one embodiment, in the process of gridding the third point cloud data after the dynamic and static point clouds are annotated by a gridding algorithm matched with the clutter identifier and the point cloud static and dynamic identifier, the point cloud data with the clutter identifier as the first identifier is determined from the point cloud data after the dynamic and static point clouds are annotated, and a plurality of fourth point cloud data are obtained; then, the plurality of fourth point cloud data are gridded to determine the grid index corresponding to each fourth point cloud data; then, static point cloud data and dynamic point cloud data are determined from the plurality of fourth point cloud data according to the point cloud static and dynamic identifier; data statistics are respectively performed on the dynamic point cloud data and the static point cloud data in each grid to obtain the number of dynamic point clouds and the number of static point clouds corresponding to each grid; then, the forward Doppler velocity of the dynamic point cloud data in each grid is accumulated to obtain the velocity accumulation value corresponding to each grid; then, according to the velocity accumulation value corresponding to each grid and the dynamic point cloud data of the corresponding grid, the velocity average value corresponding to each grid is determined; finally, based on the number of dynamic point clouds, the number of static point clouds and the velocity average value corresponding to each grid, the gridded point cloud data is generated.

[0099] For example, in Figure 2 In the process, after receiving the point cloud data, the wavelet clustering module first performs grid division on the (x, y) plane and uses SDP BRAM to create dynamic point cloud grids G mo and the static point cloud mesh G st , using TDP BRAM (TrueDual Port BRAM, true dual port BRAM) to create a dynamic point cloud Doppler grid. According to the grid size (X grid_size ,Y grid_size ) calculates the two-dimensional grid index (i, j) of each point cloud, and converts (i, j) to GridIdx through two-dimensional index-one-dimensional index mapping. GridIdx is used as the read and write address of the grid BRAM. Dynamic grid counting and Doppler accumulation are performed on the point cloud with FilterID=1 and MoveID=1, and only static grid counting is performed on the point cloud with FilterID=1 and MoveID=0. The full pipeline design ensures that in the clustering process, the grid division is completed when a frame of point cloud is received, and the dynamic grid counting matrix is ​​obtained respectively. Static Mesh Matrix Count and the dynamic grid Doppler accumulation matrix

[0100] Further, such as Figure 2 As shown, after the gridding process is completed, the wavelet clustering module needs to perform Doppler dilation, wavelet transform and connected domain search on the point cloud data to determine the cluster to which each point cloud data in the point cloud data frame belongs.

[0101] Specifically, first, Doppler dilation processing is performed on the gridded point cloud data to obtain dilated data; then, wavelet transform is performed on the gridded point cloud data to obtain target wavelet transform coefficients; then, the target wavelet transform coefficients are binarized to obtain binarized results; connected domain search is performed through the binarization results and dilated data to determine the clustering cluster corresponding to each grid; finally, the clustering cluster to which each point cloud data belongs is determined based on the correlation between the spatial coordinates corresponding to the point cloud data in each grid and the spatial coordinates of multiple processed point cloud data.

[0102] For Doppler expansion, after gridding is completed, the grids with non-zero count values ​​are read out according to formula (9): Calculate the grid Doppler average Then, the Doppler mean is expanded according to the rule shown in formula (10), the value of the Doppler mean non-zero grid is expanded to the surrounding adjacent grids, and the expanded value is Write the current Doppler grid BRAM to get Figure 6 Doppler expansion results are shown.

[0103]

[0104]

[0105] In one example, Table 1 and Table 2 respectively show the data bit definitions of the dynamic and static grid BRAMs and the data bit definitions of the Doppler expansion BRAM. In Table 1 and Table 2, the state is used to represent that the BRAM stores different data at different stages of the entire data processing flow.

[0106] Table 1

[0107]

[0108] Table 2

[0109]

[0110] In order to avoid the need to reset the BRAM for storing multi-frame point cloud data, in the embodiment of the present application, the highest bit of the BRAM data is set to the data valid flag bit Valid. Taking the grid BRAM as an example, when a new frame of point cloud data is received and meshing is performed, the BRAM data of the corresponding grid is read from (x, y) → (i, j) and the valid flag bit is judged: when Valid(i, j) = 0, the current grid is the first access to the current frame. At this time, the data output by the BRAM may be the result A(i, j) of the wavelet transform of the previous frame of point cloud. Therefore, the count value cannot be directly incremented by one, but G(i, j) = 1 is directly assigned, and Valid(i, j) = 1 is assigned; when Valid(i, j) = 1, the current grid is not the first access to the current frame. Therefore, the count value G(i, j) output by the BRAM is valid, so the self-increment operation is directly performed. The counting logic of the meshing stage is shown in formula (11):

[0111]

[0112] When performing row transformation, the data valid flag is also judged: when Valid(i,j) = 1, the count value of the current grid has been updated, so the data G(i,j) output by BRAM is directly stored in the sliding window; when Valid(i,j) = 0, the data of the current grid has not been updated, but may be the result of the wavelet transform of the point cloud of the previous frame, so 0 is stored in the sliding window. The logic of writing sliding window data during row transformation is shown in formula (12). When performing column transformation, the valid flag is no longer judged, and the BRAM output data A′ is directly written into the sliding window. While writing the column transformation result A to BRAM, Valid is changed to 0 to ensure that the data validity of all grids is reset when the next frame of grid division is performed, thereby avoiding the need to reset the BRAM data separately.

[0113]

[0114] The multiplexing logic of Doppler expansion BRAM is the same as above: when Valid(i,j)=0, the current grid is the first access of the current frame. At this time, the data output by BRAM may be the result of Doppler expansion of the previous frame v dilate Therefore, the accumulation operation cannot be performed directly, but v is directly assigned acc (i, j) = v, and assign Valid(i, j) = 1; when Valid(i, j) = 1, the current grid is not the first access of the current frame, so the Doppler accumulated value v output by BRAM acc (i, j) is valid, so the accumulation operation is performed directly. The Doppler accumulation calculation logic is shown in formula (13):

[0115]

[0116] When performing Doppler expansion, the data valid flag is also judged: when Valid(i D ,j D )=1 when i D , The Doppler accumulation value of the current grid has been updated, so the data v output by BRAM is used directly acc (i D ,j D ) to calculate the average value; when Valid(i D ,j D )=0, the data of the current grid is not updated, but may be the result of Doppler expansion of the previous frame, so 0 is directly written back to BRAM. While performing Doppler expansion, Valid is changed to 0 to ensure that the data validity of all grids is reset when the next frame of grid division is performed. The Doppler expansion logic is shown in equation (14), where (i di ,j di ) is the expanded grid index:

[0117]

[0118] In formula (14), when Valid(i D ,j D )=1, (i di ,j di )=[(max(0,i D -1),max(0,j D -1)),(min(i D +1,m-1),min(j D +1,n-1))]; when Valid(i D ,j D )=0, (i di ,j di )=(i,j).

[0119] It should be noted that the multiplication operation of the convolution calculation uses the FPGA's DSP (Digital Signal Processing) resources, and the reuse of row transformation and column transformation can save the use of DSP resources. During the column transformation, the DSP input SW is the result A' of the row transformation, and the data format is a signed number S11Q10 (that is, the signed data includes a 1-bit sign bit, a 10-bit integer bit, and a 10-bit decimal). The grid count value G is an unsigned number U10 (that is, the unsigned data includes a 10-bit integer bit). Therefore, before the row transformation, G is expanded, and the 10-bit decimal bit is expanded at the end and a 1-bit sign bit 0 is added to expand it to a signed number S11Q10, thereby realizing the reuse of the convolution calculation.

[0120] like Figure 2 As shown in FIG. 1 , after the gridding process, the FPGA performs a wavelet transform on the gridded point cloud data to obtain the following Figure 7 The wavelet transform results are shown.

[0121] Specifically, first, create a sliding window buffer with the same size as the preset wavelet basis; then, based on the sliding window buffer and the preset convolution kernel, perform row transformation on the number of dynamic point clouds and the number of static point clouds to obtain initial wavelet transform coefficients; then, perform column transformation on the initial wavelet transform coefficients to obtain target wavelet transform coefficients.

[0122] In the embodiment of the present application, the FPGA can perform a stationary wavelet transform (SWT) on the gridded point cloud data, wherein the wavelet transform coefficient A m×n The calculation can be shown as formula (15) and (16):

[0123]

[0124] In formulas (15) and (16), K l×1 is the wavelet basis vector.

[0125] For example, biorthogonal wavelet transform can be performed on dynamic grid and static grid in parallel according to formulas (15) and (16). First, row transform is performed, and the grid BRAM is traversed row by row to read out G m×n , create a l×1 sliding window buffer of the same size as the wavelet basis, perform pipeline convolution calculation on the convolution kernel and the data in the buffer, and calculate the result A′ m×n Write back to the current BRAM. Then perform column transformation and traverse the grid BRAM column by column to read A′ m×n Store it in the sliding window and perform pipeline convolution with the convolution kernel to obtain A m×n .

[0126] In the embodiment of the present application, wavelet transform can be realized by multiplexing grid BRAM and pipeline design of convolution calculation.

[0127] Specifically, first, the number of dynamic point clouds and the number of static point clouds are stored in a sliding window buffer to obtain sliding window data; then, parallel multiplication operations are performed on the sliding window data and the convolution kernel to obtain multiple multiplication data; and then parallel summation operations are performed on the multiple multiplication data to obtain the initial wavelet transform coefficients.

[0128] For example, Figure 8 The schematic diagram of the pipeline design of grid BRAM reuse and convolution calculation is shown. Figure 8 It can be seen that the data read out of BRAM during convolution calculation is stored in a sliding window composed of a shift register. l×1 With convolution kernel K l×1 Perform multiplication operations in parallel to obtain M l×1 , M l×1 The convolution result P(3) is finally obtained by performing the sum operation on the elements in parallel. In the grid division stage, the grid count matrix G is stored in the BRAM. When the row is transformed, G is read out and the operation result A′ is stored in the BRAM. When the column is transformed, the data A′ is read out and the convolution result A is stored in the BRAM. In addition, an additional cycle is used to judge the wavelet transform result A and the density threshold D. th The relationship between the convolution operations and the convolution result (i.e., the binarization result) flag are obtained.

[0129] Furthermore, if Figure 2 As shown, after obtaining the binarization result, a connected domain search is required. Specifically, after obtaining the binarization result and dilation data of the target point cloud data in the target grid, when the binarization result indicates that the wavelet transform density is valid, the dilation data is greater than the preset data, and the cluster identifier corresponding to the target point cloud data is not assigned, the grid index corresponding to the target grid is written into the connected domain search queue; then, the grid index corresponding to the target grid is obtained from the connected domain search queue, and the grid index of the adjacent grid of the target grid is calculated; after obtaining the binarization result, dilation data and cluster identifier corresponding to the grid index of the adjacent grid, when the binarization result of the adjacent grid indicates that the wavelet transform density is valid, the difference between the dilation data corresponding to the adjacent grid and the dilation data corresponding to the target grid is less than the preset difference, and the cluster identifier corresponding to the adjacent grid is not assigned, it is determined that the target grid and the adjacent grid belong to the same connected domain, and the cluster identifier corresponding to the target grid is determined to be the cluster identifier corresponding to the adjacent grid.

[0130] In the above embodiment, a breadth-first search algorithm can be used to search for connected domains for both dynamic and static data, thereby obtaining Fig. 9The connected domain search result is shown. First, traverse the grid BRAM and Doppler expansion BRAM row by row to read out the binary result MASK m×n 、ClusterID m×n and When formula (17) is satisfied, that is, the current grid wavelet transform density is valid, ClusterID is not assigned, and the Doppler expansion value v dilate >0, write ClusterID+1 into the current grid BRAM and store the current grid index (i C ,j C ) is written into the connected domain search queue FIFO.

[0131]

[0132] Then, start reading the index (i C ,j C ),i C , According to the rule of formula (18), the index of the adjacent grid (i ad ,j ad ),i ad , Then read out the MASK(i ad ,j ad ), ClusterID(i ad ,j ad ) and v dilate (i ad ,j ad ), and read out v dilate (i C ,j C ), when the grid data meets the condition of formula (19), determine (i ad ,j ad ) and (i C ,j C ) grids are the same connected domain, and ClusterID(i ad ,j ad )=ClusterID(i C ,j C ), and (i ad ,j ad ) is written into the FIFO, where v conn_th is the upper limit of the grid Doppler difference that satisfies the connected domain condition.

[0133]

[0134]

[0135] When the FIFO is not empty, repeat the following process: read the index (i ad ,j ad ), calculate the adjacent grid index according to formula (18), and determine whether the connectivity condition is met according to formula (19), complete the update of ClusterID, until the ClusterID of the current connected domain has been updated and the FIFO is read empty, perform row traversal again, and determine whether the condition of formula (17) is met, and repeat the above process. When the last grid is traversed, the search for the connected domain ends.

[0136] Finally, the point cloud data is read out from the point cloud storage BRAM again, and the grid index is used again to calculate (i, j). According to the dynamic and static point cloud identifier MoveID, the ClusterID is read out from the dynamic and static grid BRAM respectively, and the ClusterID data bit of the point cloud data is updated accordingly. The output point cloud data is (x, y, z, R, v,sinθ,MoveID,FilterID,ClusterID).

[0137] This completes the introduction of the method proposed in the embodiment of the present application.

[0138] From the above content, it can be seen that in this application, a method for point cloud dynamic and static discrimination is implemented based on FPGA Doppler statistical histogram, BRAM is used to implement Doppler statistical histogram, and cumulative sum is used to solve Doppler quantiles to calculate the dynamic and static discrimination threshold. In addition, the method provided in the embodiment of the present application uses the full on-chip resources of FPGA to implement three point cloud preprocessing functions: Doppler compensation, dynamic and static discrimination based on quantiles, and filtering of penetrable targets and ground clutter, and realizes wavelet clustering of two-dimensional space dimension + Doppler dimension based on Doppler expansion.

[0139] In the method provided in the embodiment of the present application, the computational optimization and resource reuse of wavelet clustering can also be realized on the entire FPGA chip, specifically realizing BRAM resource reuse, data valid flag to avoid BRAM reset requirements, convolution calculation reuse design of row transformation and column transformation, and data format design. The BRAM reuse and reset avoidance method based on the data valid flag saves on-chip BRAM resources while avoiding the reset operation requirements caused by repeated use of BRAM for multiple frames of data, thereby reducing the time consumption of processing each frame of data.

[0140] Finally, in this application, grid clustering similar to xyv three-dimensional grid division is achieved through xy two-dimensional grid division, the Doppler feature of the grid is embedded in the connected domain judgment condition, and the clustering effect similar to the three-dimensional grid is achieved with the two-dimensional grid, and the computational complexity is reduced from three dimensions to two dimensions. Moreover, in this application, the Doppler dimension is also used as the judgment condition for the connected domain search instead of directly as the dimension of grid division, and the design method of grid connected domain search is realized based on FPGA breadth-first search, and the connected grid index queue is realized using FIFO, thereby avoiding clustering of the three-dimensional grid, reducing the time complexity while ensuring the xyv clustering effect.

[0141] The present application also provides a millimeter wave radar point cloud data processing device, which is applied to a field programmable gate array FPGA. Fig.10 As shown, the device 1000 includes: a point cloud receiving module 1001, a first processing module 1002, a point cloud identification module 1003, a second processing module 1004 and a clustering module 1005.

[0142] The point cloud receiving module 1001 is used to sequentially receive a plurality of point cloud data included in the point cloud data frame;

[0143] The first processing module 1002 is used to perform Doppler compensation processing and clutter marking processing on multiple point cloud data in parallel to obtain multiple processed point cloud data;

[0144] The point cloud discrimination module 1003 is used to determine the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame according to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold when the point cloud data frame is received;

[0145] The second processing module 1004 is used to perform dynamic and static point cloud annotation and gridding processing on the multiple processed point cloud data in parallel through the clutter identification and the dynamic and static point cloud discrimination threshold to obtain the point cloud data frame after gridding processing, wherein the clutter identification is used to indicate whether the point cloud data contains ground clutter;

[0146] The clustering module 1005 is used to perform wavelet clustering on the gridded point cloud data frame to determine the cluster to which each point cloud data in the point cloud data frame belongs, wherein the cluster is used to characterize the target to which the corresponding point cloud data belongs.

[0147] In one example, the first processing module cyclically executes the following steps until the Doppler compensation processing and clutter annotation processing of all point cloud data in the point cloud data frame are completed to obtain multiple processed point cloud data: Doppler compensation processing is performed on the first point cloud data to obtain compensated first point cloud data, wherein the first point cloud data is any one of the multiple point cloud data; clutter annotation processing is performed on the compensated first point cloud data, and Doppler compensation processing is performed on the second point cloud data in parallel, wherein the second point cloud data is the next point cloud data of the first point cloud data; clutter annotation processing is performed on the compensated second point cloud data, and Doppler compensation processing is performed on the next point cloud data of the second point cloud data in parallel.

[0148] In one example, the compensated first point cloud data includes at least a forward Doppler velocity, and the first processing module is further used to obtain the Doppler velocity and the azimuth cosine value corresponding to the first point cloud data; query the inverse value corresponding to the azimuth cosine value from a preset data table, wherein the preset data table includes the azimuth cosine value, the inverse value of the azimuth cosine value, and the correspondence between the azimuth cosine value and the inverse value; calculate the product of the Doppler velocity and the inverse value of the azimuth cosine value to obtain the forward Doppler velocity.

[0149] In one example, the point cloud discrimination module includes: a scene acquisition module, a parameter determination module, a mapping module, a quantile determination module and a threshold determination module. The scene acquisition module is used to obtain the detection scene where the point cloud data detected by the millimeter wave radar is located; the parameter determination module is used to determine the coefficient and offset that match the detection scene; the mapping module is used to perform Doppler interval mapping on the forward Doppler velocity corresponding to each point cloud data in the point cloud data frame to obtain the interval index of the forward Doppler velocity in the Doppler statistical distribution table, wherein the Doppler statistical distribution table is a statistical distribution table constructed by block storage BRAM, and the interval index includes: the read and write address of the forward Doppler velocity in the Doppler statistical distribution table; the quantile determination module is used to determine the lower quantile and upper quantile of the forward Doppler velocity according to the interval index; the threshold determination module is used to determine the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame according to the coefficient, offset, upper quantile and lower quantile.

[0150] In one example, the quantile determination module is specifically used to obtain the data length and quantile probability of the point cloud data frame; calculate the product of the data length and the quantile probability to obtain a target count value; traverse the Doppler statistical distribution table, and accumulate the number of point cloud data corresponding to each index interval according to a preset traversal order to obtain a cumulative value; when the cumulative value is greater than or equal to the target count value, determine that the forward Doppler velocity corresponding to the point cloud data in the current index interval is the lower quantile, and the upper quantile corresponding to the lower quantile.

[0151] In one example, the first processing module is further used to detect whether the spatial coordinates corresponding to the first point cloud data are located in the target three-dimensional space, wherein the target three-dimensional space is a three-dimensional space that contains point cloud data that can pass through the target but does not contain ground clutter; when the spatial coordinates corresponding to the first point cloud data are located in the target three-dimensional space, the clutter identifier corresponding to the first point cloud data is marked as a first identifier, and the first identifier is used to characterize that the first point cloud data does not contain ground clutter; when the spatial coordinates corresponding to the first point cloud data are located outside the target three-dimensional space, the clutter identifier corresponding to the first point cloud data is marked as a second identifier, and the second identifier is used to characterize that the first point cloud data contains ground clutter.

[0152] In one example, the second processing module cyclically executes the following steps until the dynamic and static point annotation and gridding processing of all point cloud data in the multiple processed point cloud data are completed to obtain the gridded point cloud data frame: the third point cloud data is annotated with dynamic and static point clouds through the dynamic and static point cloud discrimination threshold to obtain the point cloud dynamic and static identification corresponding to the third point cloud data, wherein the third point cloud data is any one of the multiple processed point cloud data; the third point cloud data after the dynamic and static point cloud annotation is gridded through a gridding algorithm matching the clutter identification and the point cloud dynamic and static identification, and the next point cloud of the third point cloud data is annotated with dynamic and static point clouds in parallel.

[0153] In one example, the second processing module is also used to determine the point cloud data with the clutter identifier as the first identifier from the point cloud data after the dynamic and static point clouds are annotated, and obtain multiple fourth point cloud data; perform gridding processing on the multiple fourth point cloud data to determine the grid index corresponding to each fourth point cloud data; determine the static point cloud data and the dynamic point cloud data from the multiple fourth point cloud data according to the point cloud dynamic and static identifiers; perform data statistics on the dynamic point cloud data and the static point cloud data in each grid respectively to obtain the number of dynamic point clouds and the number of static point clouds corresponding to each grid; accumulate the forward Doppler velocity of the dynamic point cloud data in each grid to obtain the velocity accumulation value corresponding to each grid; determine the velocity average value corresponding to each grid according to the velocity accumulation value corresponding to each grid and the dynamic point cloud data of the corresponding grid; and generate gridded point cloud data based on the number of dynamic point clouds, the number of static point clouds and the velocity average value corresponding to each grid.

[0154] In one example, the clustering module includes: an expansion module, a transformation module, a binarization module, a connected domain search module, and a cluster determination module. The expansion module is used to perform Doppler expansion processing on the gridded point cloud data to obtain expansion data; the transformation module is used to perform wavelet transformation on the gridded point cloud data to obtain target wavelet transformation coefficients; the binarization module is used to perform binarization processing on the target wavelet transformation coefficients to obtain binarization results; the connected domain search module is used to perform connected domain search through the binarization results and the expansion data to determine the cluster corresponding to each grid; the cluster determination module is used to determine the cluster to which each point cloud data belongs based on the correlation between the spatial coordinates corresponding to the point cloud data in each grid and the spatial coordinates of multiple processed point cloud data.

[0155] In one example, the transformation module is specifically used to create a sliding window buffer with the same size as a preset wavelet basis; based on the sliding window buffer and a preset convolution kernel, the number of dynamic point clouds and the number of static point clouds are transformed in rows to obtain initial wavelet transform coefficients; the initial wavelet transform coefficients are transformed in columns to obtain target wavelet transform coefficients.

[0156] In one example, the transformation module is also used to store the number of dynamic point clouds and the number of static point clouds in a sliding window buffer to obtain sliding window data; perform parallel multiplication operations on the sliding window data and the convolution kernel to obtain multiple multiplication data; perform parallel summation operations on the multiple multiplication data to obtain initial wavelet transform coefficients.

[0157] In one example, a connected domain search module is specifically used to obtain a binarization result and dilation data of target point cloud data in a target grid; when the binarization result indicates that the wavelet transform density is valid, the dilation data is greater than the preset data, and the cluster identifier corresponding to the target point cloud data is not assigned, the grid index corresponding to the target grid is written into a connected domain search queue; the grid index corresponding to the target grid is obtained from the connected domain search queue, and the grid indexes of adjacent grids of the target grid are calculated; the binarization result, dilation data and cluster identifier corresponding to the grid index of the adjacent grid are obtained; when the binarization result of the adjacent grid indicates that the wavelet transform density is valid, the difference between the dilation data corresponding to the adjacent grid and the dilation data corresponding to the target grid is less than the preset difference, and the cluster identifier corresponding to the adjacent grid is not assigned, it is determined that the target grid and the adjacent grid belong to the same connected domain, and the cluster identifier corresponding to the target grid is determined to be the cluster identifier corresponding to the adjacent grid.

[0158] The millimeter-wave radar point cloud data processing device provided in the embodiment of the present application can implement each process implemented in the aforementioned method embodiment, and will not be described again here to avoid repetition.

[0159] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0160] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0161] The electronic device may include a processor 1101 and a memory 1102 storing computer program instructions.

[0162] Specifically, the processor 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0163] The memory 1102 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1102 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1102 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1102 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1102 is a non-volatile solid-state memory.

[0164] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0165] The processor 1101 implements any one of the millimeter-wave radar point cloud data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 1102 .

[0166] In one example, the electronic device may further include a communication interface 1103 and a bus 1110. Fig.11 As shown, the processor 1101, the memory 1102, and the communication interface 1103 are connected via a bus 1110 and communicate with each other.

[0167] The communication interface 1103 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0168] Bus 1110 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1110 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0169] In addition, in combination with the method for processing millimeter wave radar point cloud data in the above embodiments, the present application embodiment may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the methods for processing millimeter wave radar point cloud data in the above embodiments is implemented.

[0170] In addition, in combination with the method for processing millimeter wave radar point cloud data in the above embodiments, the present application embodiment may provide a computer program product for implementation. When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device executes and implements any one of the methods for processing millimeter wave radar point cloud data in the above embodiments.

[0171] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0172] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0173] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0174] The above describes various aspects of the present disclosure with reference to the flowchart and / or block diagram of the method and device for processing millimeter wave radar point cloud data according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0175] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for processing millimeter wave radar point cloud data, characterized in that: Applied in a field programmable gate array FPGA, the method comprises: receiving a plurality of point cloud data contained in the point cloud data frame in sequence; Performing Doppler compensation processing and clutter marking processing on the plurality of point cloud data in parallel to obtain a plurality of processed point cloud data; When the point cloud data frame is received, the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame is determined according to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold; Performing dynamic and static point cloud annotation and gridding processing on the plurality of processed point cloud data in parallel through the clutter identifier and the dynamic and static point cloud discrimination threshold to obtain a point cloud data frame after gridding processing, wherein the clutter identifier is used to indicate whether the point cloud data contains ground clutter; Wavelet clustering is performed on the gridded point cloud data frame to determine the cluster to which each point cloud data in the point cloud data frame belongs, wherein the cluster is used to characterize the target to which the corresponding point cloud data belongs.

2. The method according to claim 1, characterized in that Performing Doppler compensation processing and clutter marking processing on the plurality of point cloud data in parallel to obtain a plurality of processed point cloud data, including: The following steps are executed in a loop until the Doppler compensation processing and clutter marking processing of all point cloud data in the point cloud data frame are completed to obtain the plurality of processed point cloud data: Performing Doppler compensation processing on the first point cloud data to obtain compensated first point cloud data, wherein the first point cloud data is any one of the multiple point cloud data; Performing clutter annotation processing on the compensated first point cloud data, and performing Doppler compensation processing on the second point cloud data in parallel, wherein the second point cloud data is the next point cloud data of the first point cloud data; The compensated second point cloud data is subjected to clutter annotation processing, and Doppler compensation processing is performed on the next point cloud data of the second point cloud data in parallel.

3. The method according to claim 2, characterized in that The compensated first point cloud data at least includes a forward Doppler velocity, wherein Doppler compensation is performed on the first point cloud data to obtain the compensated first point cloud data, including: Obtaining the Doppler velocity and azimuth cosine value corresponding to the first point cloud data; Querying a reciprocal value corresponding to the azimuth cosine value from a preset data table, wherein the preset data table includes the azimuth cosine value, the reciprocal value of the azimuth cosine value, and a corresponding relationship between the azimuth cosine value and the reciprocal value; The forward Doppler velocity is obtained by calculating the product of the Doppler velocity and the reciprocal value of the azimuth cosine value.

4. The method according to any one of claims 1 to 3, characterized in that According to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold, determining the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame includes: Obtain the detection scene where the point cloud data detected by the millimeter wave radar is located; Determining coefficients and offsets that match the detection scene; Performing Doppler interval mapping on the forward Doppler velocity corresponding to each point cloud data in the point cloud data frame to obtain an interval index of the forward Doppler velocity in a Doppler statistical distribution table, wherein the Doppler statistical distribution table is a statistical distribution table constructed by a block storage BRAM, and the interval index includes: a read and write address of the forward Doppler velocity in the Doppler statistical distribution table; Determining a lower quantile and an upper quantile of the forward Doppler velocity according to the interval index; A target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame is determined according to the coefficient, the offset, the upper quantile, and the lower quantile.

5. The method according to claim 4, characterized in that Determining the lower quantile and the upper quantile of the forward Doppler velocity according to the interval index includes: Obtaining the data length and the quantile probability of the point cloud data frame; Calculate the product of the data length and the quantile probability to obtain a target count value; Traversing the Doppler statistical distribution table, accumulating the number of point cloud data corresponding to each index interval according to a preset traversal order, and obtaining an accumulated value; When the accumulated value is greater than or equal to the target count value, the forward Doppler velocity corresponding to the point cloud data in the current index interval is determined as the lower quantile, and the upper quantile corresponding to the lower quantile.

6. The method according to claim 2, characterized in that The clutter annotation process is performed on the compensated first point cloud data, including: Detecting whether the spatial coordinates corresponding to the first point cloud data are located in a target three-dimensional space, wherein the target three-dimensional space is a three-dimensional space that includes point cloud data that can pass through the target but does not include ground clutter; In a case where the spatial coordinates corresponding to the first point cloud data are located in the target three-dimensional space, marking the clutter identifier corresponding to the first point cloud data as a first identifier, where the first identifier is used to indicate that the first point cloud data does not contain ground clutter; When the space coordinates corresponding to the first point cloud data are outside the target three-dimensional space, the clutter identifier corresponding to the first point cloud data is marked as a second identifier, and the second identifier is used to indicate that the first point cloud data contains ground clutter.

7. The method according to claim 6, characterized in that The method of performing dynamic and static point cloud labeling and gridding processing on the plurality of processed point cloud data in parallel by using the clutter identification and the dynamic and static point cloud discrimination threshold to obtain a point cloud data frame after gridding processing includes: The following steps are executed in a loop until the dynamic and static point marking and gridding processing of all point cloud data in the plurality of processed point cloud data are completed to obtain the gridded point cloud data frame: Performing dynamic and static point cloud annotation on the third point cloud data by using the dynamic and static point cloud discrimination threshold to obtain a point cloud dynamic and static identifier corresponding to the third point cloud data, wherein the third point cloud data is any one of the multiple processed point cloud data; The third point cloud data after the dynamic and static point cloud annotation is gridded by a gridding algorithm matched with the clutter identifier and the point cloud dynamic and static identifier, and the next point cloud of the third point cloud data is marked with dynamic and static point cloud in parallel.

8. The method according to claim 7, characterized in that The third point cloud data after the dynamic and static point clouds are annotated is gridded by a gridding algorithm matched with the clutter identifier and the point cloud dynamic and static identifier, including: Determine the point cloud data whose clutter identifier is the first identifier from the point cloud data after the dynamic and static point clouds are annotated, and obtain a plurality of fourth point cloud data; Performing gridding processing on the plurality of fourth point cloud data to determine a grid index corresponding to each fourth point cloud data; Determine static point cloud data and dynamic point cloud data from the plurality of fourth point cloud data according to the point cloud static and dynamic identifiers; Performing data statistics on the dynamic point cloud data and the static point cloud data in each grid, respectively, to obtain the number of dynamic point clouds and the number of static point clouds corresponding to each grid; Accumulating the forward Doppler velocity of the dynamic point cloud data in each grid to obtain a velocity accumulation value corresponding to each grid; Determine the average speed value corresponding to each grid according to the speed accumulation value corresponding to each grid and the dynamic point cloud data of the corresponding grid; The gridded point cloud data is generated based on the number of dynamic point clouds, the number of static point clouds and the average speed corresponding to each grid.

9. The method according to claim 8, characterized in that Performing wavelet clustering on the gridded point cloud data frame to determine the cluster to which each point cloud data in the point cloud data frame belongs includes: Performing Doppler expansion processing on the gridded point cloud data to obtain expansion data; Performing wavelet transform on the gridded point cloud data to obtain target wavelet transform coefficients; Binarization is performed on the target wavelet transform coefficients to obtain a binarization result; Searching for connected domains through the binarization result and the expansion data to determine the cluster corresponding to each grid; The cluster to which each point cloud data belongs is determined according to the association relationship between the spatial coordinates corresponding to the point cloud data in each grid and the spatial coordinates of the plurality of processed point cloud data.

10. The method according to claim 9, characterized in that Performing wavelet transform on the gridded point cloud data to obtain target wavelet transform coefficients includes: Create a sliding window buffer with the same size as the preset wavelet basis; Performing row transformation on the number of dynamic point clouds and the number of static point clouds based on the sliding window buffer and a preset convolution kernel to obtain initial wavelet transformation coefficients; The initial wavelet transform coefficients are subjected to column transformation to obtain target wavelet transform coefficients.

11. The method according to claim 10, characterized in that Based on the sliding window buffer and the preset convolution kernel, the number of dynamic point clouds and the number of static point clouds are subjected to row transformation to obtain initial wavelet transformation coefficients, including: The number of dynamic point clouds and the number of static point clouds are stored in the sliding window buffer to obtain sliding window data; Performing parallel multiplication operations on the sliding window data and the convolution kernel to obtain a plurality of multiplication data; A parallel summation operation is performed on the plurality of multiplication data to obtain the initial wavelet transform coefficients.

12. The method according to claim 9, characterized in that The connected domain is searched through the binarization result and the expansion data to determine the cluster corresponding to each grid, including: Obtaining a binarization result of target point cloud data in a target grid and the dilation data; When the binarization result indicates that the wavelet transform density is valid, the dilation data is greater than the preset data, and the cluster identifier corresponding to the target point cloud data is not assigned a value, the grid index corresponding to the target grid is written into the connected domain search queue; Obtaining a grid index corresponding to the target grid from the connected domain search queue, and calculating grid indexes of adjacent grids of the target grid; Obtaining a binarization result, expansion data, and cluster identifier corresponding to a grid index of the adjacent grid; When the binarization result of the adjacent grid indicates that the wavelet transform density is valid, the difference between the expansion data corresponding to the adjacent grid and the expansion data corresponding to the target grid is less than a preset difference, and the cluster identifier corresponding to the adjacent grid is not assigned a value, it is determined that the target grid and the adjacent grid belong to the same connected domain, and the cluster identifier corresponding to the target grid is determined to be the cluster identifier corresponding to the adjacent grid.

13. A millimeter wave radar point cloud data processing device, characterized in that: Applied in a field programmable gate array FPGA, the device comprises: A point cloud receiving module, used for sequentially receiving a plurality of point cloud data contained in a point cloud data frame; A first processing module is used to perform Doppler compensation processing and clutter marking processing on the plurality of point cloud data in parallel to obtain a plurality of processed point cloud data; A point cloud discrimination module is used to determine the target dynamic and static point cloud discrimination threshold corresponding to the point cloud data frame according to the correlation between the compensated point cloud data and the dynamic and static point cloud discrimination threshold when the point cloud data frame is received; A second processing module is used to perform dynamic and static point cloud annotation and gridding processing on the plurality of processed point cloud data in parallel through a clutter identifier and the dynamic and static point cloud discrimination threshold to obtain a point cloud data frame after gridding processing, wherein the clutter identifier is used to indicate whether the point cloud data contains ground clutter; The clustering module is used to perform wavelet clustering on the gridded point cloud data frame to determine the cluster to which each point cloud data in the point cloud data frame belongs, wherein the cluster is used to characterize the target to which the corresponding point cloud data belongs.

Images