Millimeter wave radar point cloud data processing method, device, and computer program product

Abstract

The application provides a millimeter wave radar point cloud data processing method, device and computer program product, and relates to the technical field of millimeter wave radar point cloud data processing. The method comprises the following steps: obtaining original point cloud data of a millimeter wave radar and performing preprocessing to generate preprocessed point cloud data; mapping the preprocessed point cloud data to a two-dimensional plane of a bird's eye view to generate a bird's eye view grid and mapping the preprocessed point cloud data to a distance-angle plane of a forward view to generate a forward view grid; performing preliminary feature extraction respectively to generate a bird's eye view grid feature matrix and a forward view grid feature matrix; performing local feature extraction, global feature correlation, key region identification and feature weight dynamic adjustment respectively through a feature extraction model to generate a bird's eye view feature map and a forward view feature map; performing feature fusion to generate a fusion feature map; and performing target detection, semantic segmentation and motion prediction to generate a fusion output result. The application realizes high-precision and low-cost target perception.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of millimeter-wave radar point cloud data processing technology, and more specifically, to a millimeter-wave radar point cloud data processing method, device and computer program product. Background Technology

[0002] In the field of intelligent point cloud processing technology for millimeter-wave radar, current mainstream technologies focus on solving problems related to point cloud sparsity, noise interference, and scene adaptability. These mainstream technologies mainly include traditional signal processing techniques, single AI model processing techniques, and multi-sensor fusion techniques.

[0003] Traditional signal processing techniques are centered around "clustering + filtering + rule-based decision-making," such as using the density-based clustering algorithm DBSCAN to group radar point clouds and Kalman filtering to smooth target trajectories. These techniques are characterized by low computational cost and high real-time performance, making them suitable for devices with limited computing power. However, they suffer from limited feature dimensions, making them unable to handle complex scenarios such as dense traffic or occlusion, resulting in low accuracy, typically less than 80%.

[0004] Single AI model processing techniques introduce deep learning models (such as CNN, PointNet, etc.) to process point cloud data. For example, CNN is used to rasterize the point cloud into BEV feature maps and then extract local features, or PointNet is used to directly process unstructured point clouds. Compared with traditional techniques, the accuracy is improved, with an accuracy rate of about 85%-90%, and it can learn complex features, such as target shape association. However, the model function of this technique is limited, only capable of detection or segmentation, and it is susceptible to noise interference and has poor scene adaptability. Patent document CN119540139A proposes a single CNN for millimeter-wave radar point cloud processing. In this scheme, the CNN cannot associate with distant targets, such as the trajectory association between vehicles in front and behind; it is prone to misjudgment, such as misjudging a stationary guardrail as a vehicle; and the accuracy decreases in noisy environments. Patent document CN120195652A proposes a lightweight AI millimeter-wave radar personnel perception. This scheme has a narrow target range, can only detect people, and cannot identify vehicles or obstacles; the misjudgment rate is high in occluded scenes, such as missing detection when people and luggage overlap; and there is no risk assessment, so downstream modules cannot judge the reliability of the results.

[0005] Multi-sensor fusion technology combines millimeter-wave radar with cameras and lidar. For example, camera images are used to supplement the "texture features" of radar point clouds to distinguish vehicles from pedestrians, while lidar is used to supplement point cloud density to solve the radar sparsity problem. Graph neural networks (GNNs) are used to achieve cross-sensor feature fusion, resulting in high accuracy (>92%) even in complex scenes. However, this technology relies on multiple hardware components, is costly, and complex to install. Cameras are susceptible to lighting / weather conditions and are prone to failure in rain and fog. LiDAR is expensive and difficult to promote on a large scale. Patent document CN117872352A proposes a radar-camera fusion target detection scheme, which requires a camera + radar + calibration equipment, costing 2-3 times more than a single radar and exhibiting poor environmental adaptability. Summary of the Invention

[0006] To address the aforementioned problems in the prior art, in a first aspect, embodiments of the present invention propose a millimeter-wave radar point cloud data processing method. The method includes: acquiring raw point cloud data from a millimeter-wave radar and preprocessing it to generate preprocessed point cloud data; mapping the preprocessed point cloud data onto a two-dimensional plane with a bird's-eye view to generate a bird's-eye view raster; mapping the preprocessed point cloud data onto a distance-angle plane with a forward view to generate a forward-facing raster; and performing preliminary feature extraction on both the bird's-eye view raster and the forward-facing raster to generate a bird's-eye view raster feature matrix and a forward-facing raster feature matrix. The bird's-eye view raster feature matrix and the forward graph raster feature matrix are processed by a feature extraction model to perform local feature extraction, global feature association, key region identification, and dynamic adjustment of feature weights, respectively, to generate a bird's-eye view feature map and a forward graph feature map. The bird's-eye view feature map and the forward graph feature map are then fused by a feature fusion model to generate a fused feature map. Target detection, semantic segmentation, and motion prediction are performed based on the fused feature map, and a fused output result is generated based on the target detection result, semantic segmentation result, and motion prediction result.

[0007] In some implementations, preliminary feature extraction of the bird's-eye view grid and the forward view grid includes: extracting multi-dimensional feature information from the bird's-eye view grid and the forward view grid, wherein the multi-dimensional feature information includes multiple features such as point cloud density, target velocity, reflection intensity, dispersion, and temporal consistency.

[0008] In some implementations, generating a bird's-eye view feature map and a forward image feature map by performing local feature extraction, global feature association, key region identification, and dynamic adjustment of feature weights on the bird's-eye view raster feature matrix and the forward image raster feature matrix respectively through a feature extraction model includes: performing local feature extraction on the bird's-eye view raster feature matrix and the forward image raster feature matrix through a convolutional neural network in the feature extraction model; capturing the global association of features in the bird's-eye view raster feature matrix and the forward image raster feature matrix through a Transformer structure in the feature extraction model; focusing on and identifying key regions through a spatial attention mechanism in the feature extraction model; and evaluating the global importance of different feature dimensions and dynamically adjusting the feature weights of the feature dimensions through a channel attention mechanism in the feature extraction model.

[0009] In some implementations, the process of generating the bird's-eye view feature map and the forward view feature map by performing local feature extraction, global feature association, key region identification, and dynamic adjustment of feature weights on the bird's-eye view raster feature matrix and the forward view raster feature matrix respectively through a feature extraction model further includes filling in the sparse feature missing regions of the point cloud in the bird's-eye view raster feature matrix and the forward view raster feature matrix.

[0010] In some implementations, the feature fusion model is used to fuse the bird's-eye view feature map and the forward view feature map to generate a fused feature map. This includes: unifying the size of the bird's-eye view feature map and the forward view feature map by feature sampling; merging the dual-view information of the unified-size bird's-eye view feature map and the forward view feature map by feature stitching; extracting cross-view correlation features from the dual-view information by convolutional fusion; and eliminating the spatial deviation of the cross-view correlation features by feature alignment to generate the fused feature map.

[0011] In some implementations, acquiring and preprocessing the raw point cloud data from millimeter-wave radar includes: caching multiple frames of the raw point cloud data; performing time-series fusion on the cached multiple frames to generate time-series fused data; adjusting the filtering parameters for the time-series fused data based on environmental noise and / or point cloud density; removing some noise and outliers from the time-series fused data; and supplementing the data in low-density areas of the time-series fused data.

[0012] In some implementations, adjusting the filtering parameters for the time-series fused data based on ambient noise and / or point cloud density includes: increasing the distance threshold and search radius when the ambient noise ratio exceeds an ambient noise ratio threshold; increasing the number of neighboring point detections when the point cloud density is higher than a density threshold; and increasing the point cloud data intensity weight when the surface noise exceeds a surface noise threshold.

[0013] In some embodiments, the method further includes: generating a fusion confidence score of the target detection result, semantic segmentation result, and motion prediction result; and incorporating the fusion confidence score into the fusion output result.

[0014] In a second aspect, embodiments of the present invention provide a millimeter-wave radar point cloud data processing device, the device including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the millimeter-wave radar point cloud data processing method described in any of the above embodiments.

[0015] In a third aspect, embodiments of the present invention provide a computer program product including computer-readable instructions that, when executed by a processor, perform the steps of the millimeter-wave radar point cloud data processing method described in any of the above embodiments.

[0016] The millimeter-wave radar point cloud data processing method, device, and computer program product proposed in this invention realize a multi-dimensional feature extraction and target perception scheme for millimeter-wave radar point clouds based on a hybrid AI model. Employing BEV+FV dual-view raster mapping, it can simultaneously capture the global spatial distribution and longitudinal contour details of the target, compensating for the insufficient feature dimensions of a single viewpoint. This addresses the technological gap in meeting the perception requirements of "high precision, low cost, and flexible adaptation" across multiple scenarios.

[0017] The embodiments of the present invention support a multi-scenario feature configuration template library, which can be quickly adapted to multiple fields such as autonomous driving, security, and industry through software parameter adjustments, thereby reducing cross-scenario deployment costs.

[0018] The embodiments of the present invention integrate object detection, semantic segmentation and motion prediction in parallel output, and complete multi-dimensional perception through a single inference, thereby reducing the complexity and latency of module integration in downstream systems.

[0019] The millimeter-wave radar point cloud data processing technology proposed in the embodiments of this invention can be applied to specific products including: 1. Autonomous driving, such as millimeter-wave radar sensing modules for L2-L4 level autonomous vehicles, and sensing equipment for autonomous mining trucks / heavy trucks; 2. Security detection, such as perimeter security radar systems, indoor security monitoring terminals, and crowd monitoring equipment for large events; 3. Industrial detection, such as industrial equipment fault monitoring systems and foreign object detection equipment for production lines; 4. Life detection or micro-motion sensing, such as medical vital sign monitoring equipment, earthquake / building collapse search and rescue equipment, etc. Attached Figure Description

[0020] The above and other objects, features, and advantages of embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein:

[0021] Figure 1 A flowchart of a millimeter-wave radar point cloud data processing method according to an embodiment of the present invention is shown;

[0022] Figure 2 A schematic diagram of raw point cloud data preprocessing according to an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram illustrating an example of the processing of a feature extraction model according to an embodiment of the present invention is shown;

[0024] Figure 4 A schematic diagram of feature fusion processing according to an embodiment of the present invention is shown.

[0025] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0026] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way.

[0027] In one aspect, embodiments of the present invention provide a method for processing millimeter-wave radar point cloud data. (See reference...) Figure 1 The diagram illustrates a flowchart of a millimeter-wave radar point cloud data processing method 100 according to an embodiment of the present invention. The method 100 includes steps S110-S170.

[0028] In step S110, the raw point cloud data from the millimeter-wave radar is acquired and preprocessed to generate preprocessed point cloud data. The raw point cloud data is a set of discrete points generated by the radar. Preprocessing optimizes the raw radar data, and the preprocessed point cloud data may include parameters such as range, angle, and velocity.

[0029] As one embodiment of the present invention, step S110 may include steps S111-S115.

[0030] In step S111, multiple frames of the original point cloud data are cached. This step prepares for subsequent multi-frame fusion.

[0031] In step S112, the cached multi-frame data is temporally fused to generate temporally fused data. This step integrates multi-frame information, improving point cloud density.

[0032] Optionally, radar measurement data can also be labeled at the same time during this step, that is, basic parameters such as distance, angle, and velocity of the original point cloud can be recorded, and the signal-to-noise ratio can be monitored in real time, and low-quality data can be labeled.

[0033] In step S113, the filtering parameters for the time-series fused data are adjusted based on environmental noise and / or point cloud density. The current environmental noise level is calculated based on the point cloud data, and the parameters of the filtering algorithm are automatically adjusted according to the current environmental noise level.

[0034] As one embodiment of the present invention, adjusting the filtering parameters for time-series fusion data based on environmental noise and / or point cloud density may include: increasing the distance threshold and search radius when the environmental noise ratio exceeds the environmental noise ratio threshold, i.e., when the environmental noise ratio is too large, thereby preventing sparse regions from being mistakenly deleted; increasing the number of neighboring point detections when the point cloud density is higher than the density threshold, i.e., when the point cloud density is high; and increasing the point cloud data intensity weight when the surface noise exceeds the surface noise threshold, i.e., when the surface noise is too large, to ensure edge validity.

[0035] In step S114, some noise and outliers in the time-series fused data are removed. For example, statistical algorithms can be used to remove some noise and outliers.

[0036] In step S115, data in low-density regions of the time-series fusion data is supplemented. For example, the internal data between multiple points is appropriately increased based on the normal vectors of existing high-intensity point cloud data; radar physical characteristics such as divergence and scanning angle are evaluated, and sparse regions of the point cloud caused by physical characteristics are supplemented.

[0037] The following steps, S120 and S130, involve spatial grid construction, which is the generation stage of the feature carrier. During spatial grid construction, for example, using the radar installation location as the origin, the processed point cloud data is mapped onto a preset two-dimensional spatial grid. The grid size is determined based on the radar's minimum detection range and the processor's computing power. Two-dimensional grid construction involves dividing a two-dimensional planar image into a regular grid composed of countless small squares, discretizing the point cloud data in continuous space, and statistically or computationally calculating the feature information within each small square, thereby forming a structured, image-like two-dimensional feature map.

[0038] The coordinate system rules of the raster are, for example, a two-dimensional coordinate system with the radar as the origin, the x-axis pointing forward, and the y-axis pointing to the right, which determines the spatial reference for subsequent raster mapping.

[0039] In step S120, the preprocessed point cloud data is mapped onto a two-dimensional plane of the bird's eye view (BEV) to generate a bird's eye view (BEV) raster.

[0040] In step S130, the preprocessed point cloud data is mapped onto the distance-angle plane of the front view (FV) to generate a front view (FV) raster.

[0041] Steps S120 and S130 above convert the point cloud into a structured raster through dual-view (BEV+FV) mapping. The structured raster products from the two views, such as the BEV raster being a "grid in the xy plane" and the FV raster being a "grid of distance and angle", reflect the spatial characteristics of the point cloud.

[0042] In step S140, preliminary feature extraction is performed on the Bird's-eye View (BEV) raster and the Forward View (FV) raster, generating feature matrices for the BEV and FV raster respectively. This step constructs multi-dimensional feature information for each raster. Features (such as density, velocity, reflection intensity, etc.) are extracted from the BEV and FV rasters respectively, and the features of the two types of rasters are fused and calculated to enrich the information dimensions.

[0043] As an embodiment of the present invention, step S140 may include: extracting multi-dimensional feature information from the bird's-eye view grid and the forward view grid respectively, wherein the multi-dimensional feature information includes multiple of point cloud density, target velocity, reflection intensity, dispersion and time consistency.

[0044] Optionally, the multidimensional feature information may include all feature dimensions from point cloud density, target velocity, reflection intensity, dispersion, and temporal consistency. The embodiments of this invention construct a 5-dimensional feature matrix of "density + velocity + reflection intensity + dispersion + temporal consistency," which can fully exploit the physical characteristics of radar point clouds and improve the accuracy of target classification in complex scenarios.

[0045] In step S150, the feature extraction model is used to perform local feature extraction, global feature association, key area identification and dynamic adjustment of feature weights on the bird's-eye view raster feature matrix and the forward view raster feature matrix, respectively, to generate the bird's-eye view feature map and the forward view feature map.

[0046] refer to Figure 3 This diagram illustrates an example of the processing of a feature extraction model according to an embodiment of the present invention. In this example, step S150 may include steps S151-S155.

[0047] In step S151, local features are extracted from the bird's-eye view raster feature matrix and the forward raster feature matrix using the convolutional neural network (CNN) in the feature extraction model, for example, capturing the details of the point cloud distribution within the raster.

[0048] In step S152, sparse feature gaps in the point cloud within the bird's-eye view raster feature matrix and the forward view raster feature matrix are filled in, performing incomplete block processing. It should be noted that step S152 is optional. In practical applications, step S152 can be performed or omitted depending on the specific scenario and requirements.

[0049] In step S153, the global correlation of features in the bird's-eye view raster feature matrix and the forward view raster feature matrix is ​​captured through the Transformer structure in the feature extraction model, such as the target trajectory relationship between different rasters.

[0050] In step S154, the key regions, or target regions, are focused on and identified using the spatial attention mechanism in the feature extraction model. The spatial attention mechanism focuses on the key regions in the raster image.

[0051] In step S155, the global importance of different feature dimensions (or different feature channels, such as density / velocity / intensity) is evaluated through the channel attention mechanism in the feature extraction model, and the feature weights of the feature dimensions are dynamically adjusted. For example, the proportion of velocity features in dynamic targets is increased.

[0052] In steps S154 and S155 above, the spatial attention mechanism and the channel attention mechanism work together to enable the model to learn independently based on the current scene context and dynamically generate the optimal feature weight allocation.

[0053] The above steps S151-S155 employ a hybrid architecture of "CNN+Transformer+Dual Attention" (hybrid AI model) to perform multidimensional feature extraction and joint analysis on the data in the current two-dimensional spatial grid, outputting "BEV feature map" and "FV feature map" to complete the deep extraction of single-view features.

[0054] The above-described embodiments of the present invention employ a CNN+Transformer hybrid AI (Artificial Intelligence) architecture and a dual attention mechanism, which takes into account both local feature extraction and global correlation capture, while dynamically adapting feature weights to the scene to enhance the model's anti-interference ability.

[0055] In step S160, the feature fusion model is used to fuse the bird's-eye view feature map and the forward view feature map to generate a fused feature map. This step fuses the BEV and FV feature maps in multiple steps, enhances information complementarity, and realizes dual-view feature fusion.

[0056] refer to Figure 4This diagram illustrates a feature fusion process according to an embodiment of the present invention. As one embodiment of the present invention, step S160 may include steps S161-S164.

[0057] In step S161, the bird's-eye view feature map and the forward view feature map are made to the same size through feature sampling.

[0058] In step S162, the dual-view information of the bird's-eye view feature map and the forward view feature map after being unified in size is merged by feature stitching.

[0059] In step S163, cross-view correlation features are extracted from dual-view information through convolutional fusion.

[0060] In step S164, spatial biases of associated features across different viewpoints are eliminated through feature alignment, generating a fused feature map. The generated fused feature map possesses the advantages of dual-view features, namely the global spatial distribution of the BEV and the longitudinal contour details of the FV.

[0061] In step S170, target detection, semantic segmentation, and motion prediction are performed based on the fused feature map. A fused output result is generated based on the target detection result, semantic segmentation result, and motion prediction result. For example, this step inputs the fused unified feature map into the task-specific decoder header for parallel analysis, and finally outputs structured judgment results such as target detection boxes, semantic segmentation maps, and future trajectories.

[0062] This step, based on the fused feature map, simultaneously performs three tasks: object detection, semantic segmentation, and motion prediction, achieving multi-task output. Object detection can include identifying object types, such as vehicles / pedestrians, and outputting bounding boxes. Semantic segmentation can distinguish between the object and the background region. Motion prediction can include predicting the trajectory of the object, such as vehicles changing lanes or pedestrians crossing the road.

[0063] Finally, the results of the three types of tasks are integrated into a "fusion output result" as the final output of the processing method. This final output can be used, for example, to be passed to an autonomous driving decision-making system.

[0064] The above steps integrate object detection, semantic segmentation, and motion prediction into parallel output, achieving multi-dimensional perception through a single inference, thus reducing the complexity and latency of module integration in downstream systems.

[0065] The millimeter-wave radar point cloud data processing method proposed in the embodiments of the present invention adopts BEV+FV dual-view grid mapping, which can simultaneously capture the global spatial distribution and longitudinal contour details of the target, making up for the deficiency of insufficient feature dimensions of a single view.

[0066] The embodiments of the present invention introduce multi-frame temporal fusion and low-density grid interpolation to improve the density and integrity of millimeter-wave radar point clouds and solve the detection omission problem caused by sparse point clouds.

[0067] As one embodiment of the present invention, the method may further include: generating a fusion confidence score of the target detection result, semantic segmentation result, and motion prediction result; and incorporating the fusion confidence score into the fusion output result. For example, a fusion confidence score is generated during the decoding process, and a confidence score is set for the output content to determine whether the object is trustworthy, providing a confidence reference for the downstream decision-making and planning module.

[0068] Subsequently, the results of AI analysis, such as target type, posture, behavior, and confidence level, can be output to downstream modules, such as decision-making systems for autonomous driving and alarm devices for security.

[0069] The millimeter-wave radar point cloud data processing method proposed in this invention realizes a multi-dimensional feature extraction and target perception scheme for millimeter-wave radar point clouds based on a hybrid AI model, which can fill the technological gap of "high precision, low cost, and flexible adaptation" perception requirements in multiple scenarios. The applicable scenarios for this invention include: 1. Autonomous driving: achieving all-weather, high-precision perception without hardware dependence; 2. Security and industrial inspection: lightweight design that flexibly adapts to multiple scenarios; 3. Life detection or micro-motion perception: achieving contactless, highly robust micro-motion perception.

[0070] On the other hand, embodiments of the present invention provide a millimeter-wave radar point cloud data processing device, the device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, implements the millimeter-wave radar point cloud data processing method described in any of the above embodiments.

[0071] In another aspect, embodiments of the present invention provide a computer program product including computer-readable instructions that, when executed by a processor, perform the steps of the millimeter-wave radar point cloud data processing method described in any of the above embodiments.

[0072] The millimeter-wave radar point cloud data processing method, device, and computer program product proposed in this invention realize a multi-dimensional feature extraction and target perception scheme for millimeter-wave radar point clouds based on a hybrid AI model. Employing BEV+FV dual-view raster mapping, it can simultaneously capture the global spatial distribution and longitudinal contour details of the target, compensating for the insufficient feature dimensions of a single viewpoint. This addresses the technological gap in meeting the perception requirements of "high precision, low cost, and flexible adaptation" across multiple scenarios.

[0073] The embodiments of the present invention support a multi-scenario feature configuration template library, which can be quickly adapted to multiple fields such as autonomous driving, security, and industry through software parameter adjustments, thereby reducing cross-scenario deployment costs.

[0074] The embodiments of the present invention integrate object detection, semantic segmentation and motion prediction in parallel output, and complete multi-dimensional perception through a single inference, thereby reducing the complexity and latency of module integration in downstream systems.

[0075] The millimeter-wave radar point cloud data processing technology proposed in the embodiments of this invention can be applied to the following specific products: 1. Autonomous driving, such as millimeter-wave radar sensing modules for L2-L4 level autonomous vehicles, and sensing equipment for autonomous mining trucks / heavy trucks; 2. Security detection, such as perimeter security radar systems, indoor security monitoring terminals, and crowd monitoring equipment for large events; 3. Industrial detection, such as industrial equipment fault monitoring systems and foreign object detection equipment for production lines; 4. Life detection or micro-motion sensing, such as medical vital sign monitoring equipment, earthquake / building collapse search and rescue equipment, etc.

[0076] The foregoing description of embodiments of the invention has been given for illustrative purposes and is not exhaustive, nor is it intended to limit the invention to the exact forms disclosed. Those skilled in the art will understand that various changes can be made without departing from the scope of the invention, and elements therein can be substituted with equivalents. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of the invention without departing from the basic scope of the invention. Therefore, the invention is not intended to be limited to the specific embodiments disclosed as the best mode contemplated for carrying out the invention; the invention will include all embodiments falling within the scope of the appended claims.

Claims

1. A method for processing millimeter-wave radar point cloud data, characterized in that, The method includes: Acquire raw point cloud data from millimeter-wave radar and preprocess it to generate preprocessed point cloud data; The preprocessed point cloud data is mapped onto a two-dimensional plane from a bird's-eye view to generate a bird's-eye view raster. The preprocessed point cloud data is mapped onto the distance-angle plane of the forward view to generate a forward map raster. Preliminary feature extraction is performed on the bird's-eye view raster and the forward view raster respectively to generate the bird's-eye view raster feature matrix and the forward view raster feature matrix; Using a feature extraction model, local feature extraction, global feature association, key region identification, and dynamic adjustment of feature weights are performed on the bird's-eye view raster feature matrix and the forward view raster feature matrix, respectively, to generate bird's-eye view feature map and forward view feature map; The feature fusion model is used to fuse the bird's-eye view feature map and the forward view feature map to generate a fused feature map. Target detection, semantic segmentation, and motion prediction are performed based on the fused feature map. A fused output result is generated based on the target detection results, semantic segmentation results, and motion prediction results. Specifically, through a feature extraction model, local feature extraction, global feature association, key region identification, and dynamic adjustment of feature weights are performed on the bird's-eye view raster feature matrix and the forward view raster feature matrix, respectively, to generate the bird's-eye view feature map and the forward view feature map, including: Local features are extracted from the bird's-eye view raster feature matrix and the forward view raster feature matrix using the convolutional neural network in the feature extraction model. The global correlation between features in the bird's-eye view raster feature matrix and the forward view raster feature matrix is ​​captured by the Transformer structure in the feature extraction model. The spatial attention mechanism in the feature extraction model is used to focus on and identify key regions. The global importance of different feature dimensions is evaluated through the channel attention mechanism in the feature extraction model, and the feature weights of the feature dimensions are dynamically adjusted.

2. The method according to claim 1, characterized in that, Preliminary feature extraction is performed on the bird's-eye view grid and the forward view grid, respectively, including: Multidimensional feature information is extracted from the bird's-eye view grid and the forward view grid, respectively. The multidimensional feature information includes multiple features such as point cloud density, target velocity, reflection intensity, dispersion, and temporal consistency.

3. The method according to claim 1, characterized in that, The process of generating the bird's-eye view feature map and the forward view feature map by performing local feature extraction, global feature association, key region identification, and dynamic adjustment of feature weights on the bird's-eye view raster feature matrix and the forward view raster feature matrix respectively through the feature extraction model further includes: Fill in the sparse feature gaps in the point cloud in the bird's-eye view raster feature matrix and the forward view raster feature matrix.

4. The method according to any one of claims 1-3, characterized in that, By using a feature fusion model, the bird's-eye view feature map and the forward view feature map are fused to generate a fused feature map, including: By sampling features, the bird's-eye view feature map and the forward view feature map are made to the same size; By stitching features together, the dual-view information of the bird's-eye view feature map and the forward view feature map after unifying the size is merged. By using convolutional fusion, cross-view correlation features are extracted from the dual-view information; By aligning features, spatial biases in the cross-view associated features are eliminated, and the fused feature map is generated.

5. The method according to any one of claims 1-3, characterized in that, Acquiring and preprocessing raw point cloud data from millimeter-wave radar includes: Cache multiple frames of raw point cloud data; Perform time-series fusion on cached multi-frame data to generate time-series fused data; Adjust the filtering parameters for the time-series fused data based on environmental noise and / or point cloud density; Remove some noise and outliers from the time-series fused data; Supplement the data in low-density areas of the time-series fusion data.

6. The method according to claim 5, characterized in that, Adjusting the filtering parameters for the time-series fused data based on environmental noise and / or point cloud density includes: If the ambient noise ratio exceeds the ambient noise ratio threshold, increase the distance threshold and the search radius; When the point cloud density is higher than the density threshold, increase the number of neighboring point detections; When the surface noise exceeds the surface noise threshold, increase the point cloud data intensity weight.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: Generate a fusion confidence score for object detection results, semantic segmentation results, and motion prediction results; The fusion confidence score is incorporated into the fusion output result.

8. A millimeter-wave radar point cloud data processing device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the millimeter-wave radar point cloud data processing method according to any one of claims 1-7.

9. A computer program product comprising computer-readable instructions, characterized in that, When the instruction is executed by the processor, the steps of the millimeter-wave radar point cloud data processing method according to any one of claims 1-7 are performed.

Images