A point cloud imaging and positioning method based on chip-level millimeter-wave radar

Abstract

The present invention discloses a point cloud imaging and positioning method based on a chip-level millimeter wave radar, including: receiving echo information and performing windowing processing on each frame of echo signal, then performing FFT of fast time dimension and slow time dimension to obtain multiple frames of RDM; merging multiple frames of RDM, and then setting coarse-grained labels using a low threshold; extracting azimuth and pitch angles for each frame of RDM, and converting the target point cloud set to a Cartesian coordinate system for clustering and denoising; fine-grained labeling of RDM according to the denoised point cloud; using each frame of RDM and RDM with fine-grained labels as input and output, respectively, and training a deep learning network to obtain an RDM target detection network; obtaining multiple frames of RDM of a target to be measured during the movement of a mobile radar, extracting point cloud data after being sent to the RDM target detection network for processing, and then performing registration to obtain the position and posture of the radar at each frame and the final point cloud image of the target to be measured. The present invention can efficiently generate high-quality and dense point cloud images.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar positioning, and in particular relates to a point cloud imaging and positioning method based on a chip-level millimeter-wave radar. Background Art

[0002] Traditional millimeter-wave radars mostly output point cloud-level information. Although this information is obtained through processing of the original echo signal, some details will inevitably be lost during the conversion process, resulting in incomplete point cloud data. In order to improve the quality of point clouds and enhance the accuracy of target detection, researchers have begun to focus on the development and application of chip-level millimeter-wave radar technology. By directly processing target echo information at the chip level, the loss of information during transmission and processing is reduced, thereby achieving higher-quality imaging and more accurate target detection. The exploration of this technical path has not only brought innovation to millimeter-wave radar technology itself, but also laid a solid foundation for the future development of intelligent driving and robotics technology.

[0003] The CFAR radar point cloud detector is a method based on a statistical model. If this method is used directly to detect the range-Doppler map, range-azimuth map or range-elevation angle map without considering radar characteristics such as clutter and multipath effects, it will cause the radar to generate a large number of false targets. The point cloud quality is reduced. Some targets occupy multiple cells, and this method will cause the target to be missed, the point cloud density will be reduced, and it will pose a greater challenge to the subsequent point cloud processing.

[0004] The millimeter-wave radar range-Doppler map (RDM) is obtained by fast Fourier transforming the original echo signal in two dimensions: fast time and full time. It represents the distance and Doppler velocity of the target, not the three-dimensional image or point cloud information of the target. Therefore, how to use RDM to efficiently and quickly generate point cloud data is a huge challenge.

[0005] At present, the commonly used point cloud matching algorithms include the iterative closest point (ICP) algorithm, the normal distribution transform (NDT) algorithm and the algorithm based on ICP improvement. These algorithms have shown good performance in laser point cloud matching. However, there are great differences in point cloud density, point cloud precision and point cloud accuracy between millimeter wave radar point cloud and laser radar point cloud. Therefore, the performance of the algorithm will be greatly reduced when these traditional point cloud algorithms are used to match millimeter wave radar point cloud. Summary of the invention

[0006] The present invention provides a point cloud imaging and positioning method based on chip-level millimeter-wave radar, which can more efficiently generate high-quality and dense point cloud images.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] A point cloud imaging and positioning method based on chip-level millimeter-wave radar, comprising:

[0009] Step 1: Receive the echo information and perform windowing on each frame of the echo signal, then perform FFT in the fast time dimension and the slow time dimension to obtain the range-Doppler velocity map of the millimeter wave radar, which is recorded as one frame of RDM;

[0010] Step 2, accumulating multiple frames of RDM obtained at the same location to obtain an accumulated RDM, and then setting a coarse-grained label for the accumulated RDM using a preset low power threshold; the label is used to mark whether each grid in the accumulated RDM includes the target point;

[0011] Step 3: extract the azimuth and elevation angle of the target from each frame of RDM at the same location, and then transform the point cloud of the target into a Cartesian coordinate system by combining the distance of the target, and perform clustering and denoising on the multi-frame point cloud of the target in the Cartesian coordinate system;

[0012] Step 4: Align the RDM with coarse-grained labels with the point cloud set obtained in step 3, and perform fine-grained labeling on the RDM with coarse-grained labels according to the point cloud set: if a certain grid of the RDM corresponds to point cloud data, the fine-grained label of the grid is marked as having a target point, otherwise it is marked as having no target point, and finally the RDM with fine-grained labels is obtained;

[0013] Step 5, using the RDM of each frame obtained in step 1 and the RDM with fine-grained labels obtained in step 4 as the input and output of the deep learning network, respectively, and training the deep learning network to obtain the RDM target detection network;

[0014] Step 6, moving the radar and obtaining multiple RDM frames of the target to be detected during the movement, sending each RDM frame to the RDM target detection network to obtain the corresponding RDM target detection result;

[0015] Step 7: According to the RDM target detection results, the azimuth and elevation angles of the target are extracted respectively, and multi-frame point cloud data of the target to be detected are constructed;

[0016] Step 8: align the multi-frame point cloud data of the target to be measured to obtain the position and posture of the radar at each frame and the final point cloud image of the target to be measured.

[0017] Furthermore, the pitch angle of the target is extracted by using a multivariate signal classification method, specifically:

[0018] Perform FFT transformation on the echo signal and then calculate the covariance matrix:

[0019]

[0020] In the formula, is the number of snapshots, It is the signal obtained by FFT of the echo signal. is the covariance matrix, represents the transpose operation;

[0021] Covariance matrix Perform eigenvalue decomposition to obtain the signal subspace and the noise subspace :

[0022]

[0023] In the formula, is the matrix composed of the first k eigenvalues ​​with the largest values ​​obtained from eigenvalue decomposition, and the matrix composed of the remaining eigenvalues ​​is recorded as ; The number of eigenvalues ​​k in is determined by the number of sources in the echo signal obtained by azimuth fast Fourier transform;

[0024] Constructing spatial spectrum function using noise subspace :

[0025]

[0026] In the formula, is the steering vector, determined by the radar hardware;

[0027] By traversing all angles The angle with the largest peak value of the spatial spectrum function is found, which is the pitch angle of the target.

[0028] Furthermore, in step 3, the DBSCAN algorithm is used to perform clustering and denoising on the multi-frame point cloud of the target in a Cartesian coordinate system.

[0029] Furthermore, the deep learning network adopts a U-Net framework, and uses a GAN network to optimize and train the parameters in the U-Net.

[0030] Furthermore, the registering of multiple frames of point cloud data of the target to be measured includes:

[0031] Step 8.1: Get the distance accuracy of the millimeter wave radar , Pitch angle accuracy and horizontal angle accuracy , construct the precision covariance matrix :

[0032]

[0033] Step 8.2, according to the precision covariance matrix , calculate each point cloud The error matrix , and then calculate each point cloud Weight :

[0034]

[0035]

[0036] In the formula, Point cloud The pitch angle and horizontal angle, Reason The three-dimensional rotation matrix formed;

[0037] Step 8.3, use the iterative closest point algorithm to match the point clouds of different frames and obtain the radar pose; the matching model constructed is:

[0038]

[0039] In the formula, and The radar is The rotation matrix and translation matrix at the frame together constitute the radar at the The pose at the time of the frame; and Point cloud In the Frame and The coordinates of the frame in the radar local coordinate system, For the Frame and The number of points in common in the frame.

[0040] Furthermore, the registering of the multi-frame point cloud data of the target to be measured further includes:

[0041] Step 8.4, cluster the point cloud data registered in step 8.3 using the DBSCAN algorithm to find the noise points, then discard the noise points from the unregistered point cloud data of each frame, and repeat steps 8.1 to 8.3 for the unregistered point cloud data of the current frame.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The traditional CFAR radar point cloud detector is based on the grid neighborhood value for judgment. This method will be interfered by noise. In addition, if the target and edge target are easily filtered out, the present invention is based on the deep network learning method. The RDM target detection network (MMRPCGNet) can be trained without the help of other sensors. The point cloud density is increased by lowering the RDM threshold, and then the clutter points are removed by the DBSCAN algorithm. The density of the point cloud generated will be improved.

[0044] 2. By improving the traditional ICP algorithm, the weight based on radar measurement accuracy is introduced to improve the accuracy of point cloud matching;

[0045] 3. The two-stage point cloud matching method of coarse matching and fine matching is adopted to improve the positioning accuracy, while improving the point cloud imaging quality and reducing clutter points and outliers. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the point cloud imaging and positioning method based on chip-level millimeter-wave radar described in an embodiment of the present application;

[0047] Figure 2 This is the registration flow chart described in the embodiment of this application. DETAILED DESCRIPTION

[0048] The following is a detailed description of an embodiment of the present invention. This embodiment is based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process to further explain the technical solution of the present invention.

[0049] This embodiment provides a point cloud imaging and positioning method based on chip-level millimeter wave radar. Figure 1 As shown, the following steps are included.

[0050] Step 1: Receive the echo information and perform windowing on each frame of the echo signal, then perform FFT in the fast time dimension and the slow time dimension to obtain the range-Doppler velocity map of the millimeter wave radar, which is recorded as one frame of RDM.

[0051] Step 2, accumulate multiple frames of RDM obtained at the same location to obtain a cumulative RDM, and then use a preset low power threshold to set a coarse-grained label for the cumulative RDM; the label is used to mark whether each grid in the cumulative RDM includes the target point.

[0052] The present invention combines multiple frames of RDM obtained by radar at the same location into one RDM for processing, which can improve the signal-to-noise ratio of the target therein, and further improve the accuracy of judging whether each grid is a target.

[0053] Step 3: Extract the azimuth and pitch angle of the target from each frame RDM at the same location, and then transform the point cloud of the target into a Cartesian coordinate system in combination with the distance of the target, and perform clustering and denoising on the multi-frame point cloud of the target in the Cartesian coordinate system.

[0054] This embodiment uses FFT to estimate the azimuth angle and uses the multivariate signal classification (MUSIC) method to extract the pitch angle of the target. The MUSIC method is as follows:

[0055] (1) Perform FFT transformation on the echo signal and then calculate the covariance matrix:

[0056]

[0057] In the formula, is the number of snapshots, It is the signal obtained by FFT of the echo signal. is the covariance matrix, Represents the transpose operation.

[0058] (2) Covariance matrix Perform eigenvalue decomposition to obtain the signal subspace and the noise subspace :

[0059]

[0060] In the formula, is the matrix composed of the first k eigenvalues ​​with the largest values ​​obtained from eigenvalue decomposition, and the matrix composed of the remaining eigenvalues ​​is recorded as ; The number of eigenvalues ​​k in the signal subspace is determined by the number of sources in the echo signal obtained by azimuth fast Fourier transform; yes The eigenvectors corresponding to the first k eigenvalues ​​with the largest values ​​obtained by eigenvalue decomposition constitute the noise subspace, and the eigenvectors corresponding to the remaining eigenvalues ​​constitute the noise subspace. .

[0061] (3) Using noise subspace to construct spatial spectrum function :

[0062]

[0063] In the formula, is the steering vector, determined by the radar hardware;

[0064] (4) By traversing all angles The angle with the largest peak value of the spatial spectrum function is found, which is the pitch angle of the target.

[0065] The range-Doppler velocity map RDM includes distance and velocity information. The above method is used to extract the azimuth and pitch angle of the target, that is, the four-dimensional information of each point cloud of the target is obtained, which is recorded as . Then transform each point cloud into the Cartesian coordinate system:

[0066]

[0067] Repeat the above process for each frame of RDM at the same location to obtain more point clouds. Since the RDM is obtained at the same location, the radar poses used to obtain the above frame point clouds are the same, and the point cloud data of the same location can be directly merged and clustered. In this embodiment, the DBSCAN algorithm is used in the Cartesian coordinate system to cluster and denoise the multi-frame point clouds of the target to obtain a new point cloud set.

[0068] In step 4, the RDM with coarse-grained labels is aligned with the point cloud set obtained in step 3, and the RDM with coarse-grained labels is fine-grainedly labeled according to the point cloud set: if a certain grid of the RDM corresponds to point cloud data, the fine-grained label of the grid is marked as having a target point, otherwise it is marked as having no target point, and finally an RDM with fine-grained labels is obtained.

[0069] The Cartesian point cloud obtained in step 3 is converted into the accumulated RDM in step 2. If a point cloud is marked as a target in step 2, it is retained, otherwise it is discarded.

[0070] In step 5, the RDM of each frame obtained in step 1 and the RDM with fine-grained labels obtained in step 4 are used as the input and output of the deep learning network, respectively, and the deep learning network is trained to obtain the RDM target detection network.

[0071] The deep learning network in this embodiment adopts the U-Net framework, and uses the GAN network to optimize the parameters in the U-Net to obtain the RDM target detection network. The specific training method is:

[0072] First, the RDM of each frame obtained in step 1 and the RDM with fine-grained labels obtained in step 4 are used as the input and output of the deep learning network, respectively, and the parameters in the standard U-Net are trained to obtain a pre-trained U-Net network;

[0073] Then, the RDM of each frame obtained in step 1 is input into the pre-trained U-Net network, and the corresponding output is the RDM with the predicted label;

[0074] Then the RDM with predicted labels is input into the GAN network, and the RDM with higher precision labels is output;

[0075] Finally, the RDM of each frame obtained in step 1 and the RDM with higher precision labels are used as input and output respectively, and the pre-trained U-Net network is further trained to obtain the final RDM target detection network.

[0076] The deep learning network in this embodiment improves the original U-Net framework and utilizes the generative adversarial network (GAN) network to constrain the predicted label matrix (U-Net segmentation result) to improve the accuracy of the predicted label matrix.

[0077] Step 6, moving the radar and obtaining multiple RDM frames of the target to be detected during the movement, sending each RDM frame to the RDM target detection network to obtain the corresponding RDM target detection result.

[0078] Step 7: According to the RDM target detection results, the azimuth and elevation angles of the target are extracted respectively, and multi-frame point cloud data of the target to be detected are constructed.

[0079] Step 8: align the multi-frame point cloud data of the target to be measured to obtain the position and posture of the radar at each frame and the final point cloud image of the target to be measured.

[0080] In this embodiment, registering different frames of point cloud data of the target to be measured includes:

[0081] Step 8.1: Get the distance accuracy of the millimeter wave radar , Pitch angle accuracy and horizontal angle accuracy , construct the precision covariance matrix :

[0082]

[0083] Step 8.2, according to the precision covariance matrix , calculate each point cloud The error matrix , and then calculate each point cloud Weight :

[0084]

[0085]

[0086] In the formula, Point cloud The pitch angle and horizontal angle, Reason The three-dimensional rotation matrix formed;

[0087] Step 8.3, use the iterative closest point (ICP) algorithm to roughly match the point clouds of different frames and obtain the radar pose; the matching model constructed is:

[0088]

[0089] In the formula, and The radar is The rotation matrix and translation matrix at the frame together constitute the radar at the The pose at the time of the frame; and Point cloud In the Frame and The coordinates of the frame in the radar local coordinate system, For the Frame and In this embodiment, the radar motion trajectory and point cloud image can be preliminarily obtained by solving the above matching model using least squares.

[0090] In a more preferred embodiment, the registering of the multi-frame point cloud data of the target to be measured further includes:

[0091] Step 8.4: Use the DBSCAN algorithm to cluster the point cloud data registered in step 8.3 to find the noise points, then discard the noise points from the unregistered point cloud data of each frame, and repeat steps 8.1 to 8.3 for the unregistered point cloud data of the current frame, so as to obtain a higher precision radar motion trajectory and point cloud image. Figure 2 shown.

[0092] The above embodiments are preferred embodiments of the present application. Ordinary technicians in this field can also make various changes or improvements on this basis. Without departing from the overall concept of the present application, these changes or improvements should fall within the scope of protection required by the present application.

Claims

1. A point cloud imaging and positioning method based on chip-level millimeter-wave radar, characterized in that: include: Step 1: Receive the echo information and perform windowing on each frame of the echo signal, then perform FFT in the fast time dimension and the slow time dimension to obtain the range-Doppler velocity map of the millimeter wave radar, which is recorded as one frame of RDM; Step 2, multiple frames of RDM obtained at the same location are accumulated to obtain an accumulated RDM, and then a coarse-grained label is set for the accumulated RDM using a preset low power threshold; The label is used to mark whether each grid in the cumulative RDM includes the target point; Step 3: extract the azimuth and elevation angle of the target from each frame of RDM at the same location, and then transform the point cloud of the target into a Cartesian coordinate system by combining the distance of the target, and perform clustering and denoising on the multi-frame point cloud of the target in the Cartesian coordinate system; Step 4: Align the RDM with coarse-grained labels with the point cloud set obtained in step 3, and perform fine-grained labeling on the RDM with coarse-grained labels according to the point cloud set: if a certain grid of the RDM corresponds to point cloud data, the fine-grained label of the grid is marked as having a target point, otherwise it is marked as having no target point, and finally the RDM with fine-grained labels is obtained; Step 5, using the RDM of each frame obtained in step 1 and the RDM with fine-grained labels obtained in step 4 as the input and output of the deep learning network, respectively, and training the deep learning network to obtain the RDM target detection network; Step 6, moving the radar and obtaining multiple RDM frames of the target to be detected during the movement, sending each RDM frame to the RDM target detection network to obtain the corresponding RDM target detection result; Step 7, according to the RDM target detection results, respectively extract the azimuth and pitch angle of the target, and construct multi-frame point cloud data of the target to be detected; Step 8: align the multi-frame point cloud data of the target to be measured to obtain the position and posture of the radar at each frame and the final point cloud image of the target to be measured.

2. The point cloud imaging and positioning method based on chip-level millimeter-wave radar according to claim 1 is characterized in that: The pitch angle of the target is extracted by using the multivariate signal classification method, specifically: Perform FFT transformation on the echo signal and then calculate the covariance matrix: ； In the formula, is the number of snapshots, It is the signal obtained by FFT of the echo signal. is the covariance matrix, represents the transpose operation; Covariance matrix Perform eigenvalue decomposition to obtain the signal subspace and the noise subspace : ； In the formula, is the matrix composed of the first k eigenvalues ​​with the largest values ​​obtained from eigenvalue decomposition, and the matrix composed of the remaining eigenvalues ​​is recorded as ; The number of eigenvalues ​​k in is determined by the number of sources in the echo signal obtained by azimuth fast Fourier transform; Constructing spatial spectrum function using noise subspace : ； In the formula, is the steering vector, determined by the radar hardware; By traversing all angles The angle with the largest peak value of the spatial spectrum function is found, which is the pitch angle of the target.

3. The point cloud imaging and positioning method based on chip-level millimeter-wave radar according to claim 1 is characterized in that: In step 3, the DBSCAN algorithm is used to cluster and denoise the multi-frame point cloud of the target in the Cartesian coordinate system.

4. The point cloud imaging and positioning method based on chip-level millimeter-wave radar according to claim 1, characterized in that: The deep learning network adopts the U-Net framework, and uses the GAN network to optimize the parameters in the U-Net.

5. The point cloud imaging and positioning method based on chip-level millimeter-wave radar according to claim 1, characterized in that: The registering of multiple frames of point cloud data of the target to be measured includes: Step 8.1: Get the distance accuracy of the millimeter wave radar , Pitch angle accuracy and horizontal angle accuracy , construct the precision covariance matrix : ； Step 8.2, according to the precision covariance matrix , calculate each point cloud The error matrix , and then calculate each point cloud Weight : ； ； In the formula, Point cloud The pitch angle and horizontal angle, Reason The three-dimensional rotation matrix formed; Step 8.3, use the iterative closest point algorithm to match the point clouds of different frames and obtain the radar pose; the matching model constructed is: ； In the formula, and The radar is The rotation matrix and translation matrix at the frame together constitute the radar at the The pose at the time of the frame; and Point cloud In the Frame and The coordinates of the frame in the radar local coordinate system, For the Frame and The number of points in common in the frame.

6. The point cloud imaging and positioning method based on chip-level millimeter-wave radar according to claim 5 is characterized in that: The registering of the multi-frame point cloud data of the target to be measured also includes: Step 8.4, cluster the point cloud data registered in step 8.3 using the DBSCAN algorithm to find the noise points, then discard the noise points from the unregistered point cloud data of each frame, and repeat steps 8.1 to 8.3 for the unregistered point cloud data of the current frame.

Images