Millimeter wave radar point cloud denoising method based on distance perception cross attention mechanism

Abstract

The invention discloses a millimeter wave radar point cloud denoising method based on a distance perception cross attention mechanism. A pair of visible light images and millimeter wave radar point clouds is given, a monocular depth map is generated based on the visible light images, then depth context features are extracted from the monocular depth map, and point cloud geometric features are extracted from the millimeter wave radar point clouds. Thirdly, performing deep fusion on the depth context features and the point cloud geometric features by using a distance perception cross attention mechanism; and finally, a classifier is trained based on the fusion features, effectiveness discrimination is performed on each radar point, noise points are filtered out, and millimeter wave radar point cloud after denoising enhancement is output. The invention also discloses a method for automatically generating the training label through the adaptive distance and depth threshold. According to the method, the scene depth context and the geometric structure of the point cloud can be effectively combined, accurate noise filtering of the millimeter wave radar point cloud can be realized, and cleaner and more reliable high-quality point cloud data can be obtained.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of sensor data processing and artificial intelligence, and particularly relates to a point cloud processing method for multi-modal information fusion by deep learning, in particular to a millimeter wave radar point cloud denoising method based on distance perception cross attention mechanism. BACKGROUND

[0002] In the fields of autonomous driving, robot navigation and environment perception, it is crucial to obtain accurate and reliable 3D environment information. Millimeter-wave radar has become an indispensable sensor in perception systems due to its robustness in various weather and lighting conditions, relatively low cost, and the ability to directly measure target velocity. However, compared with lidar, the point cloud data obtained by millimeter-wave radar usually has inherent limitations, mainly manifested as: sparsity, i.e., low spatial sampling density; high noise level, containing a large number of clutters, false target points, and interference points caused by multipath propagation effects; low spatial resolution, making it difficult to accurately depict the fine contours of objects. These problems severely limit the direct application of millimeter-wave radar point cloud in scenarios that require high-precision 3D information, such as fine environment modeling, small target detection and tracking, etc. To overcome the above shortcomings of millimeter-wave radar point cloud and improve its data quality, researchers have proposed various denoising and enhancement methods. Traditional methods include filters based on spatial neighborhood statistics (such as radius filtering, statistical filtering) or clustering algorithms (such as DBSCAN), but these methods are often sensitive to parameters and have limited effectiveness when dealing with point clouds of uneven density, easily misdeleting valid sparse points or retaining clustered noise points. Another class of methods attempts to accumulate and fuse multiple frames of radar data to increase point cloud density and filter out transient noise using temporal information, but this introduces additional processing delay and requires higher compensation for dynamic scenes and sensor motion. With the development of deep learning technology, using neural networks to process point clouds has become a new research direction. Some works directly apply PointNet, PointNet++, or graph neural networks (GNN) on radar point cloud for feature learning and noise classification. However, relying solely on the geometric information of the radar point cloud, the ability to distinguish between valid signals and noise is still limited when facing severe noise and sparsity. To introduce more rich environmental information, sensor fusion methods have emerged, especially combining visual information (such as RGB images or depth maps generated from them) to assist radar point cloud processing. These methods usually assume that visual sensors can provide more rich scene structure and semantic information. For example, by projecting radar points onto the image plane, combining image features or segmentation results to judge the validity of radar points. However, simple projection and two-dimensional image feature-based judgment may not work well due to calibration errors, synchronization errors, and the lack of accurate three-dimensional spatial correspondence. More advanced fusion methods attempt to fuse features at the feature level, such as concatenation or using basic attention mechanisms. Although existing sensor fusion methods have made some progress, there are still significant technical challenges. First, it is still difficult to effectively fuse different modalities of features, such as how to best combine the sparse geometric point cloud provided by radar and the relatively dense but possibly inaccurate scale depth context generated from visual information. Second, current fusion mechanisms are still insufficient in utilizing the three-dimensional spatial structure information of point cloud itself.Many methods focus on the association based on two-dimensional projection, and fail to fully exploit and utilize the geometric layout and point-to-point relationship of point cloud in real three-dimensional space. Specifically, the existing fusion strategy often ignores the key information of three-dimensional spatial distance between points when making noise discrimination at the point level, and this information is crucial for judging the context consistency of a point with its surrounding environment, and directly affects the accuracy of noise filtering. Therefore, there is an urgent need for a method that can effectively combine the geometric information of millimeter wave radar point cloud and the depth context information provided by the visual sensor, and through an advanced fusion mechanism that can perceive the three-dimensional spatial distance relationship, to realize an accurate and robust millimeter wave radar point cloud denoising method. SUMMARY

[0003] In view of the problems of noise and sparsity of millimeter wave radar point cloud in the prior art and the deficiency that the existing denoising method fails to effectively fuse multi-modal information and fully utilize the three-dimensional spatial relationship of point cloud, the present application provides a millimeter wave radar point cloud denoising method based on distance-aware cross-attention mechanism, which aims to guide the accurate denoising of sparse and noisy millimeter wave radar point cloud with the help of depth context information generated from visible light images, thereby significantly improving the quality of millimeter wave radar point cloud data. The method of the present application receives visible light images and sparse millimeter wave radar point cloud as input, first generates a monocular depth map from the visible light image using a monocular depth estimation model to obtain the depth context information of the scene, and at the same time extracts the point cloud geometric features of the millimeter wave radar point cloud itself using a graph convolutional neural network. The core is that the present application proposes and adopts a distance-aware cross-attention mechanism to deeply fuse the aforementioned depth context features and point cloud geometric features. The innovation of this mechanism is to introduce the three-dimensional spatial distance information between point clouds into the calculation process of attention weight, so that the fusion not only considers the feature similarity, but also explicitly models the spatial layout relationship of the point cloud, thereby promoting the effective guidance of the context information to the geometric feature expression. Finally, based on the high-discrimination features after the deep fusion, a classifier is trained to accurately discriminate the effectiveness of each radar point, filter out noise interference, and finally output high-quality millimeter wave radar point cloud after accurate denoising. Through the novel feature fusion method, the present application effectively combines the scene depth context and the inherent geometric features of the point cloud, and realizes the accurate filtering of millimeter wave radar noise.

[0004] The present application adopts the following technical solutions:

[0005] A millimeter wave radar point cloud denoising method based on a distance-aware cross-attention mechanism, first acquires a visible light image and generates a monocular depth map based on the visible light image to provide scene depth context information, while acquiring a corresponding sparse millimeter wave radar point cloud; then depth context features and point cloud geometric features are extracted from the monocular depth map and the millimeter wave radar point cloud respectively; then a distance-aware cross-attention mechanism is used to fuse the depth context features and the point cloud geometric features to obtain fused features, which considers the three-dimensional spatial distance information between point cloud pairs when calculating attention weights; finally, each point in the millimeter wave radar point cloud is classified based on the fused features to distinguish and filter out noise points, and the denoised millimeter wave radar point cloud is output. The millimeter wave radar point cloud denoising method based on the distance-aware cross-attention mechanism has the following specific steps:

[0006] S1: receiving a visible light image I∈R 3×H×W and a corresponding sparse millimeter wave radar point cloud where H, W are the height and width of the image respectively, N is the number of radar points, and C r is the original feature dimension of each radar point;

[0007] S2: inputting the visible light image I into a monocular depth estimation model to generate a monocular depth map D m ∈R 1×H×W ;

[0008] S3: inputting the monocular depth map D m into a ResNet feature extractor to extract depth context features where C m is the context feature dimension, and s is the network downsampling factor;

[0009] S4: inputting the millimeter wave radar point cloud P r into a graph convolutional neural network to extract point cloud geometric features where C p is the geometric feature dimension;

[0010] S5: inputting the depth context features F m and the point cloud geometric features F p into a distance-aware cross-attention fusion module to perform feature fusion to obtain fused features where C f is the fused feature dimension, and the fused features consider the three-dimensional spatial distance between points in the millimeter wave radar point cloud P r ;

[0011] S6: inputting the fused features F f into a classifier to classify Pr Classify each point in and get the classification result L∈{0,1} N , the points marked as 0 in the classification results are noise, and the points marked as 1 are valid points;

[0012] S7: Filter P according to the classification result L r The noise points in (marked as 0) are used to obtain the denoised millimeter-wave radar point cloud where N f ≤N is the number of valid points.

[0013] In the above technical solution, further, the specific operations of step S4 include:

[0014] 1) Based on the millimeter wave radar point cloud The three-dimensional space coordinates of the midpoint, construct its k

[0015] Neighborhood graph G = (V, E), where V is the point set and E is the edge set representing the k-nearest neighbor connection relationship;

[0016] 2) Millimeter wave radar point cloud P r The original features (including three-dimensional coordinates and other radar measurements) are used as the initial node features, and dynamic edge convolution is applied on the k-nearest neighbor graph G.

[0017] Perform feature propagation and aggregation on E to learn and extract point cloud geometric features containing local geometric information The original features of the millimeter-wave radar point cloud include three-dimensional coordinates and other radar measurement values.

[0018] Furthermore, the distance-aware cross-attention fusion module in step S5 is used to fuse deep context features and point cloud geometry features To obtain fusion features The specific operations of step S5 include:

[0019] 1) For the point cloud geometric feature F p Each point feature in According to the two-dimensional pixel coordinates (x, y) of the corresponding original radar point on the visible light image, the deep context feature F is obtained by bilinear interpolation. m Get the feature f of the point p The corresponding context feature vector Then the point cloud geometric feature F p The context feature vectors corresponding to all point features are combined into a context feature matrix

[0020] 2) The point cloud geometric features F pAfter linear transformation, a query matrix is obtained The context feature matrix is obtained After two independent linear transformations, a key matrix and a value matrix are obtained respectively and where d k and d v are the dimensions of the key and the value respectively;

[0021] 3) Calculate the three-dimensional Euclidean distance between any two points in the millimeter wave radar point cloud P r , and construct a three-dimensional space distance matrix D 3d ∈R N×N ; At the same time, the point cloud geometric feature F p is calculated through a linear layer to obtain the distance modulation factor corresponding to each point τ=Linear(F p ), where H attn is the number of attention heads, and Linear is a linear layer;

[0022] 4) Using the three-dimensional space distance matrix D 3d and the distance modulation factor τ, generate a distance attention mask M for each attention head dist =M 3d τ, which aims to suppress the attention between points with long spatial distance;

[0023] 5) Calculate the scaled dot-product attention score between the query matrix Q p and the key matrix K m , and modulate it with the distance attention mask M dist , specifically Then, apply the Softmax function to the modulated attention score A mod along the dimension of the key to obtain the final distance-aware attention weight

[0024] 6) Using the final distance-aware attention weight W, perform weighted summation on the value matrix V m to obtain the attention fusion feature F attn =WV m ; Through residual connection, layer normalization and subsequent multi-layer perceptron (MLP), the attention fusion feature F attn is processed together with the point cloud geometric feature F p to obtain the final fusion feature

[0025] Further, in the step S6, the millimeter wave radar point cloud P rThe classifier for classifying each point in is specifically a multi-layer perceptron network. The step S6 is specifically as follows: the multi-layer perceptron network receives the fusion feature As input, the feature vector representing each radar point is processed independently The multi-layer perceptron network outputs a classification score logit for each point (specifically a floating point number between 0 and 1, where a smaller value indicates a greater probability that the point is a noise point). Finally, the classification score logit of each point is compared with a preset threshold. Points with a score less than the threshold are considered noise points, while points with a score greater than the threshold are considered valid points. In this way, the final classification label of each point is determined, thereby forming the classification result L∈{0,1} N .

[0026] The present invention also provides a millimeter-wave radar point cloud denoising neural network model based on a distance-aware cross-attention mechanism, which is used to execute the above method.

[0027] The present invention also provides a method for generating radar point cloud classification labels (for training the neural network model), which uses high-precision lidar point cloud P l As a reference truth value, the input millimeter wave radar point cloud P r Automatically generate a validity label for each point in the , which includes the following steps:

[0028] 1) For the millimeter-wave radar point cloud P to be marked r Every point p in r , which has three-dimensional coordinates XYZ r and depth D r , in the lidar point cloud P l In the example, based on the three-dimensional space distance, find the point p r The closest k lidar points are recorded as the initial neighbor set The corresponding depths are

[0029] 2) Calculate point p r Each lidar neighbor point found in 1) The three-dimensional distance between Then, for each lidar neighbor point According to the depth of the neighboring point of the lidar Apply an adaptive distance threshold that varies with depth Determine the nearest neighbor points of the lidar Whether the spatial distance condition is met:

[0030] All the lidar neighbor points that meet this spatial distance condition are screened out to form a valid lidar neighbor point set N that passes the distance filtering.dist .

[0031] 3) For set N dist Each lidar neighbor point in (The depth is D l ), calculate its depth D l With millimeter wave radar point p r Depth D r The absolute difference between |D r -

[0032] D l |; Then, according to the laser radar neighbor point p l Depth D l , apply another adaptive depth threshold function depth_thresh(D l ), determine whether the absolute difference meets the depth consistency condition: |D r -D l |≤depth_thresh(D l ); Collect all the lidar neighboring points that meet the depth consistency conditions to form the final depth verification pass set N depth ,

[0033]

[0034] 4) According to the set N dist and N depth The number of points contained in the millimeter wave radar point p is used to determine r The final label: If | N dist |=0, indicating that there is no nearby laser radar neighbor point, then the point

[0035] p r Marked as "unknown" (label value 2); if the set N dist is not empty, and the set N depth The number of midpoints is greater than or equal to the set N dist If the number of midpoints is half, it means that the radar point has good consistency in depth with its neighboring lidar points, and it is marked as "valid" (label value 1); in all other cases (that is, there are lidar points nearby but the depth difference is large),

[0036] Then point p r Labeled as "noise" (label value is 0).

[0037] Furthermore, the adaptive distance threshold that varies with depth used in step 2) is defined as: Among them, d represents the input depth value, α dist , β dist, K dist are pre-set positive real number parameters. Usually β dist <α dist As the input depth value d increases, the distance threshold dist_thresh(d) decreases, i.e. the requirement for spatial distance becomes more stringent.

[0038] Further, the depth-varying adaptive depth threshold function depth_thresh(d) used in step 3) is defined as: where d represents the input depth value, α depth , β depth , K depth are pre-set positive real number parameters. Usually β depth >α depth As the input depth value d increases, the depth threshold depth_thresh(d) increases, i.e. the tolerance for depth difference becomes more relaxed.

[0039] The beneficial effects of the present application are:

[0040] By effectively combining the scene context information provided by the monocular depth map generated from the visible light image and the fine local geometric features of the point cloud extracted by the graph convolution network, the precision and reliability of the millimeter wave radar point cloud denoising are significantly improved, which is superior to the method relying on only a single modal information. The core advantage lies in the introduction of a distance-aware cross-attention mechanism for multi-modal feature fusion, which innovatively incorporates the three-dimensional spatial distance information between point clouds into the attention calculation, not only focusing on feature similarity, but also explicitly modeling the distribution relationship and local consistency of point clouds in the real three-dimensional space, thereby achieving more effective feature fusion and enhancing the ability to distinguish and filter noise (such as multi-path reflection points). In addition, the present application also provides a high-quality label automatic generation method based on laser radar reference and adaptive threshold, which guarantees the effectiveness of neural network model training. Finally, the present application can output cleaner and more reliable millimeter wave radar point clouds, which can support their usability and performance in downstream tasks such as target detection, tracking, mapping and positioning. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is the overall operation flowchart of the embodiment of the present application.

[0042] Figure 2 is the distance-aware cross-attention mechanism flowchart of the embodiment of the present application.

[0043] Figure 3 is the radar point cloud classification label generation method flowchart of the embodiment of the present application.

[0044] Figure 4For example results of embodiments of the present application, all example results are projected onto RGB images for visualization: a) RGB image; (b) monocular depth map; (c) noisy millimeter wave radar point cloud; (d) noisy millimeter wave radar point cloud classification result (blue points represent valid points, red crosses represent noise); (e) accuracy result of the classification result (green points represent TP, gray points represent TN, red crosses represent FP, and yellow crosses represent FN); DETAILED DESCRIPTION

[0045] The technical scheme of the present application is further illustrated below in combination with specific embodiments and drawings.

[0046] EMBODIMENT

[0047] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments herein are only used to explain the present application and not to limit the present application.

[0048] The present application provides a millimeter wave radar point cloud denoising method based on distance-aware cross-attention mechanism, the core idea of which is to effectively fuse the scene depth context information generated by visible light image and the geometric structure information of millimeter wave radar point cloud itself, to interact and enhance the features through an attention mechanism that perceives the distance in three-dimensional space, and finally to accurately filter out the noise points in the radar point cloud.

[0049] The overall operation process of the millimeter wave radar point cloud denoising method based on distance-aware cross-attention mechanism is shown in Figure 1 , which specifically includes the following steps:

[0050] 1) Data input and preprocessing: obtain a pair of synchronously collected visible light image I 3×H×W (as shown in Figure 4 (a)) and corresponding original sparse millimeter wave radar point (as shown in Figure 4 (c)), wherein P r contains the three-dimensional coordinates XYZ and other possible radar measurement features (such as reflection intensity, speed, etc.) of each point. The visible light image I is input into a pre-trained monocular depth estimation model to generate a corresponding monocular depth map D m 1×H×W (as shown in Figure 4 (b)). Although the absolute scale of this monocular depth map may not be accurate, it can provide rich scene structure details and relatively accurate depth relationships as important context information.

[0051] ​2) Depth context feature extraction: input the visible light image I into a pre-trained monocular depth estimation model (Depth Anything model) to generate a monocular depth map D m ∈R 1×H×W The depth map provides the relative depth structure information of the scene. Subsequently, the monocular depth map D m is input into an image feature extraction network (convolutional neural network based on ResNet architecture) to extract depth context features F

[0052] 3) Point cloud geometry feature extraction: input the input millimeter wave radar point cloud P r into a graph convolutional neural network (GCN). Specifically, first construct a k-neighbor graph G according to the three-dimensional coordinates of the point cloud, and then apply graph convolution operations on the graph to aggregate neighborhood information, learn and extract point cloud geometry features for each point

[0053] 4) Distance-aware cross-attention feature fusion (as shown in FIG. 4): this is the core module of the present application. The extracted depth context features F m and point cloud geometry features F p are input into the distance-aware cross-attention module. The module process is roughly as follows:

[0054] a) First, for each point feature f r in the point cloud geometry feature F m , according to the projection position of each point in P p on the image, sample the point feature f p corresponding context feature vector f p from F k Combine the context feature vectors corresponding to all point features in the point cloud geometry feature F v into a context feature matrix F r

[0055] b) Then, linearly transform the point cloud geometry feature F 3d to obtain a query matrix Q Linearly transform the context feature matrix F twice to obtain a key matrix K

[0056] and a value matrix V where d N and d ×N are the dimensions of the key and value, respectively.

[0057] c) Calculate the three-dimensional Euclidean distance between points in the original radar point cloud P p to form a three-dimensional spatial distance matrix D​3d ∈R N ×N , and the point cloud geometric features F p Calculate the distance modulation factor corresponding to each point through the linear layer τ=Linear(F p ), where H attn is the number of attention heads. The distance modulation factor suppresses the attention weights between spatially distant point pairs, strengthening the consideration of local spatial consistency.

[0058] d) Using the three-dimensional distance matrix D 3d and the distance modulation factor τ to generate a distance attention mask for each attention head M dist =D 3d τ. Calculate the query matrix Q p With the key matrix K m The scaled dot product attention score between , and apply the distance attention mask M dist Modulation, specifically Then the modulated attention score A mod Apply the Softmax function along the key dimension to obtain the final distance-aware attention weights

[0059] e) Finally, use the final distance-aware attention weight W to calculate the value matrix V m Perform weighted summation to obtain the attention fusion feature F attn =WV m ; The attention fusion feature F attn and point cloud geometric features F p Through residual connection, layer normalization and subsequent multi-layer perceptron (MLP) network processing, the fusion feature is finally obtained.

[0060] 5) Classification: Fusion feature F f Input a classifier (a simple multi-layer perceptron MLP), which classifies each radar point (valid point / noise point / unknown) and outputs the classification result L∈{0,1} for each point N The points marked as 0 are noise points, and the points marked as 1 are valid points.

[0061] To supervise the training of neural networks (especially classifiers), this embodiment uses an automatic label generation method based on lidar reference to generate validity labels (such as Figure 3 The process of this method is as follows:

[0062] a) Input the millimeter-wave radar point cloud P to be markedr and reference lidar point cloud P l .

[0063] b) Find the k nearest neighbors of p r in P r . l

[0064] c) Calculate the 3D spatial distance between p r and its k nearest neighbors.

[0065] d) Apply an adaptive distance threshold dist_thresh that varies with depth to filter out points that are not valid neighbors, resulting in a set of valid lidar neighbors N dist .

[0066] e) Calculate the depth difference between p r and each point in N dist .

[0067] f) Apply an adaptive depth threshold depth_thresh that varies with depth to filter out points that do not satisfy the depth consistency, resulting in a set of depth-validated points N depth .

[0068] g) Determine the label (valid / noise / unknown) of p dist in P depth based on the size (number of points) relationship between N r and N r .

[0069] Training and supervision of the classifier: use the validity labels generated in the above steps as ground truth to supervise the predicted results output by the classifier, calculate the loss function, and update the parameters of the entire neural network model (including the feature extractor, fusion module, and classifier) through backpropagation and optimization algorithms.

[0070] 6) Noise point filtering and output: in the inference stage, apply the classification results obtained in step 5) to the input millimeter wave radar point cloud P r , filter out points classified as noise (label 0), and finally output the high-quality millimeter wave radar point cloud P f after denoising.

[0071] Figure 4 The figure intuitively demonstrates the actual effect of the embodiment of the present application in processing millimeter wave radar point clouds in a typical driving scenario. The input data are the conventional RGB image shown in Figure 4 (a) and the original millimeter wave radar point cloud shown in Figure 4 (c), the latter of which obviously presents the characteristics of sparsity and mixed with a large number of noise points, which is a common data challenge for millimeter wave radars. The present application first utilizes Figure 4 ​The image generation of (a) is as shown in Figure 4 (b), to extract the scene structure as context information. After the denoising method based on distance-aware cross-attention proposed in the present application is processed, the point cloud classification result is as shown in Figure 4 (d). In the figure, the points judged as valid signals are marked in blue, while the points judged as noise and filtered out are marked in red crosses. It can be clearly seen that, compared with the original input Figure 4 (c)), the point cloud in the output result Figure 4 (d)) significantly reduces the cluttered noise and retains the points more consistent with the scene structure (such as the vehicle in front), preliminarily proving that the method of the present application can effectively distinguish valid points from noise. In order to more accurately evaluate the classification performance, Figure 4 (e) further shows the comparison between the classification result and the true value (obtained by the label generation method): a large number of green points (true positives TP) and gray points (true negatives TN) indicate that the method accurately identifies most of the valid points and noise points; while a small number of red crosses (false positives FP) and yellow crosses (false negatives FN) reflect the misjudgment situation. Overall, ​ (e) is the visualization result of the classification confusion matrix, which directly proves that the method of the present application can achieve high classification accuracy, effectively improves the quality of the millimeter wave radar point cloud, and verifies the superior effect of the denoising method of the present application.

[0072] The embodiment of the present application is tested on a device equipped with an NVIDIA RTX 3090 GPU, and the input image size is 1600x900. The total parameter quantity of the implemented model is about 11,422,694. The inference speed reaches about 121.09 frames per second (FPS), and the single-frame inference time is about 8.26 milliseconds. On a typical test data set, the following performance indicators are obtained: test loss (Test Loss) is 0.3932, accuracy (Accuracy) is 82.19%, precision (Precision) is 80.65%, recall (Recall) is 74.39%, F1 score (F1 Score) is 77.40%, area under ROC curve (ROCAUC) is 0.9039, and area under PR curve (PR AUC) is 0.8807. The above quantitative indicators show that the embodiment of the present application is not only lightweight, but also excellent in denoising task.

[0073] In summary, the embodiment of the present application effectively fuses the monocular depth context and the point cloud geometric features by introducing the distance-aware cross-attention mechanism, and combines the label generation method based on adaptive threshold for training, thereby realizing efficient and accurate denoising of the millimeter wave radar point cloud, and the effect is verified by the visualization result and the quantitative indicators.

Claims

1. A millimeter-wave radar point cloud denoising method based on distance-aware cross-attention mechanism, characterized in that: First, a visible light image is acquired and a monocular depth map is generated based on the visible light image to provide scene depth context information, and a corresponding millimeter-wave radar point cloud is acquired; then, depth context features and point cloud geometric features are extracted from the monocular depth map and the millimeter-wave radar point cloud respectively; Then, a distance-aware cross-attention mechanism is used to fuse the deep context features and the point cloud geometric features to obtain a fused feature. The distance-aware cross-attention mechanism considers the three-dimensional spatial distance information between point cloud pairs when calculating the attention weight. Finally, each point in the millimeter-wave radar point cloud is classified based on the fused feature to distinguish and filter out noise points, and the denoised millimeter-wave radar point cloud is output.

2. The millimeter-wave radar point cloud denoising method based on distance-aware cross-attention mechanism according to claim 1 is characterized in that: The specific steps are as follows: S1: Receive visible light image I∈R 3×H×W And the corresponding millimeter wave radar point cloud Where H, W are the height and width of the image respectively, N is the number of radar points, C r is the original feature dimension of each radar point; S2: Input the visible light image I into the monocular depth estimation model to generate a monocular depth map D m ∈R 1×H×W ; S3: The monocular depth map D m Input ResNet feature extractor to extract deep context features Among them C m is the context feature dimension, s is the network downsampling factor; S4: The millimeter wave radar point cloud P r Input graph convolutional neural network to extract point cloud geometric features Among them C p is the geometric feature dimension; S5: The deep context feature F m And the point cloud geometric features F p Input the distance-aware cross-attention fusion module to perform feature fusion to obtain fusion features Among them C f is the fusion feature dimension, and the fusion feature takes into account the millimeter wave radar point cloud P r The three-dimensional distance between midpoints; S6: The fusion feature F f Input classifier, P r Classify each point in and get the classification result L∈{0,1} N , in the classification results, the points marked as 0 are noise points, and the points marked as 1 are valid points; S7: Filter P according to the classification result L r The noise points in the denoised millimeter-wave radar point cloud are obtained where N f ≤N is the number of valid points.

3. The millimeter-wave radar point cloud denoising method based on distance-aware cross-attention mechanism according to claim 2 is characterized in that: The step S4 specifically includes the following steps: 1) Based on the millimeter wave radar point cloud The three-dimensional spatial coordinates of the midpoint are used to construct its k-nearest neighbor graph G = (V, E), where V is the point set and E is the edge set representing the k-nearest neighbor connection relationship; 2) Millimeter wave radar point cloud P r The original features of are used as the initial node features, and dynamic edge convolution is applied on the k-nearest neighbor graph G. By performing feature propagation and aggregation on the edge set E, the point cloud geometric features containing local geometric information are learned and extracted. The original features of the millimeter-wave radar point cloud include three-dimensional coordinates and other radar measurement values.

4. The millimeter-wave radar point cloud denoising method based on distance-aware cross-attention mechanism according to claim 2 is characterized in that: The step S5 specifically includes the following steps: 1) For the point cloud geometric feature F p Each point feature in According to the two-dimensional pixel coordinates (x, y) of the corresponding original radar point on the visible light image, the deep context feature F is obtained by bilinear interpolation. m Get point feature f p The corresponding context feature vector Then the point cloud geometric feature F p The context feature vectors corresponding to all point features are combined into a context feature matrix 2) The point cloud geometric features F p After linear transformation, the query matrix is ​​obtained The context feature matrix After two independent linear transformations, the bond matrix Sum Matrix where d k and d v are the dimensions of key and value respectively; 3) Calculate the millimeter wave radar point cloud P r The three-dimensional Euclidean distance between any two points in the , forming a three-dimensional space distance matrix D 3d ∈R N×N ; At the same time, the point cloud geometric features F p Calculate the distance modulation factor corresponding to each point through the linear layer τ=Linear(F p ), where H attn is the number of attention heads, Linear is the linear layer; 4) Using the three-dimensional space distance matrix D 3d and the distance modulation factor τ to generate a distance attention mask for each attention head M dist =D 3d τ; 5) Calculate the query matrix Q p With the key matrix K m The scaled dot product attention score between , and apply the distance attention mask M dist Modulation, specifically Then the modulated attention score A mod Apply the Softmax function along the key dimension to obtain the final distance-aware attention weights 6) Using the final distance-aware attention weight W to calculate the value matrix V m Perform weighted summation to obtain the attention fusion feature F attn =WV m ; The attention fusion feature F attn and point cloud geometric features F p Through residual connection, layer normalization and multi-layer perceptron processing, the fusion feature is finally obtained.

5. The millimeter-wave radar point cloud denoising method based on distance-aware cross-attention mechanism according to claim 2 is characterized in that: In step S6, the millimeter wave radar point cloud P r The classifier for classifying each point in is specifically a multi-layer perceptron network; the step S6 is specifically: the multi-layer perceptron network receives the fusion feature As input, the feature vector representing each radar point is processed independently The multi-layer perceptron network outputs a classification score logit for each point; finally, the classification score logit of each point is compared with a preset threshold to determine the final classification label of each point, thereby forming the classification result L∈{0,1} N .

6. A millimeter-wave radar point cloud denoising neural network model based on distance-aware cross-attention mechanism, characterized by: Used to perform the method according to any one of claims 1 to 5.

7. A method for generating radar point cloud classification labels for training the neural network model of claim 6, characterized in that: This method uses the lidar point cloud P l As a reference truth value, the input millimeter wave radar point cloud P r Automatically generate a validity label for each point in the , which includes the following steps: 1) For the millimeter-wave radar point cloud P to be marked r Every point p in r , which has three-dimensional coordinates XYZ r and depth D r , in the lidar point cloud P l In the example, based on the three-dimensional space distance, find the point p r The closest k lidar points are recorded as the initial neighbor set The corresponding depths are 2) Calculate point p r Each lidar neighbor point found in 1) The three-dimensional distance between Then, for each lidar neighbor point According to the depth of the neighboring point of the lidar Apply an adaptive distance threshold that varies with depth Determine the nearest neighbor points of the lidar Whether the spatial distance condition is met: All the lidar neighbor points that meet this spatial distance condition are screened out to form a valid lidar neighbor point set N that passes the distance filtering. dist ; 3) For set N dist Each lidar neighbor point in Calculate its depth D l With millimeter wave radar point p r Depth D r The absolute difference between |D r -D l |; Then, according to the laser radar neighbor point p l Depth D l , apply another adaptive depth threshold depth_thresh(D l ), determine whether the absolute difference meets the depth consistency condition: |D r - D l |≤depth_thresh(D l ); Collect all the lidar neighboring points that meet the depth consistency conditions to form the final depth verification pass set N depth , 4) According to the set N dist and N depth The number of points contained in the millimeter wave radar point p is used to determine r The final label: If | N dist |=0, indicating that there is no nearby laser radar neighbor point, then point p r Marked as "unknown", the label value is 2; if the set N dist is not empty, and the set N depth The number of midpoints is greater than or equal to the set N dist If the number of points is half of the midpoint, it is marked as "valid" with a label value of 1; in all other cases, the point p r Labeled as "noise", the label value is 0.

8. The radar point cloud label generation method according to claim 7, characterized in that: The adaptive distance threshold used in step 2) is defined as: Among them, d represents the input depth value, α dist , β dist 、 K dist are pre-set positive real number parameters, β dist <α dist , as the input depth value d increases, dist_thresh(d) decreases.

9. The radar point cloud label generation method according to claim 7, characterized in that: The adaptive depth threshold used in step 3) is defined as: Among them, d represents the input depth value, α depth , β depth , K depth is a pre-set positive real number parameter, β depth >α depth , as the input depth value d increases, depth_thresh(d) increases.

Images