3D object perception method based on dual space fusion and depth-occupancy consistency

Abstract

This invention discloses a 3D target perception method based on dual-space fusion and depth-occupancy consistency, belonging to the field of 3D target perception technology, including the following steps: S1, image multi-scale feature extraction and radar 3D point perspective projection alignment; S2, constructing a cross-modal cross-attention fusion module in the perspective domain, fusing two types of features, and outputting enhanced perspective domain fusion features; S3, constructing a depth prediction network and supervising and constraining the radar ground truth depth; S4, perspective feature cross-space transformation and dual-modal BEV feature fusion; S5, 3D detection and 3D occupancy joint prediction under BEV fusion features; S6, depth-occupancy geometric consistency constraint and total loss joint optimization. This invention adopts dual-space fusion of perspective domain + BEV domain. The perspective domain fuses radar and image features through cross-attention, and the BEV domain fuses BEV features from two types of sensors, fully exploring radar geometric priors, achieving complementary advantages, and improving the accuracy of target localization and recognition.

Description

Technical Field

[0001] This invention relates to the field of 3D target perception technology, and in particular to a 3D target perception method based on dual-space fusion and depth-occupancy consistency. Background Technology

[0002] In autonomous driving environmental perception, 3D target detection and scene understanding are core tasks for achieving safe and reliable decision-making. Current mainstream solutions are mainly divided into two categories: pure visual perception and multi-sensor fusion. Pure visual solutions rely on RGB images acquired by cameras, completing environmental modeling through depth estimation and semantic understanding. However, this method is susceptible to factors such as changes in lighting, inclement weather, and target occlusion, leading to inaccurate depth estimation and a significant decrease in the performance of long-range target detection. In contrast, millimeter-wave radar can directly measure the distance, velocity, and azimuth information of targets, offering advantages such as being unaffected by lighting and texture, having a long detection range, and adapting to inclement weather. Therefore, radar-camera fusion has become an important technical path to improve the robustness of autonomous driving perception. However, existing radar-camera fusion methods have the following shortcomings: (1) The fusion space is singular, and fusion is only performed in the BEV space. The radar prior is not used to enhance image features in the perspective domain. (2) The radar utilization rate is low, and it is only used as a point cloud feature input, without using radar depth as a monitoring signal; (3) 3D detection and 3D occupancy prediction are independent of each other and lack geometric consistency constraints, resulting in low accuracy in long distances and occluded areas; (4) The end-to-end joint learning capability is weak, and the robustness and generalization are difficult to meet the needs of complex scenarios.

[0003] To address the above issues, a 3D target perception method based on dual-space fusion and depth-occupancy consistency is proposed. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art by proposing a 3D target perception method based on dual-space fusion and depth-occupancy consistency.

[0005] A 3D target perception method based on dual-space fusion and depth-occupancy consistency includes the following steps: S1. Acquire multi-view camera images and millimeter-wave radar point clouds, and perform multi-scale feature extraction of images and radar 3D point perspective projection alignment respectively. S2. Construct a cross-modal cross-attention fusion module in the perspective domain, using radar projection features as query vectors and image features as key vectors, adaptively fusing the two types of features, and outputting the enhanced perspective domain fusion features. S3. Construct a depth prediction network based on perspective domain fusion, and combine it with radar ground truth depth to achieve supervision and constraint. S4. Perform cross-space transformation on the optimized perspective domain fusion features, and simultaneously encode the radar 3D point cloud in the BEV domain to achieve dual-modal BEV feature fusion. S5. A dual-branch parallel prediction network is built based on the BEV fusion feature tensor to achieve the joint output of 3D detection and 3D occupancy. S6. Establish depth-occupancy geometric consistency constraints, integrate the losses of each task to construct the total loss function, and achieve joint optimization of the model.

[0006] Preferably, step S1 specifically includes: We collect raw RGB image data from multi-view cameras, and use a deep convolutional backbone network to extract multi-level and multi-scale semantic features from the images. We mine the shallow texture contour features, mid-level target semantic features and deep global scene features of the images layer by layer, and output a high-dimensional dense image feature tensor. Acquire three-dimensional spatial point cloud data collected by millimeter-wave radar, and complete the coordinate transformation from radar coordinate system to camera coordinate system based on the pre-calibrated intrinsic parameter matrix and extrinsic parameter transformation matrix of radar and camera; By using a perspective projection geometry model, all three-dimensional spatial points of the radar are mapped onto the two-dimensional pixel plane of the camera, achieving precise alignment and matching between the radar point cloud and image pixels in spatial dimensions.

[0007] Preferably, step S2 specifically includes: A cross-modal cross-attention fusion mechanism is constructed using radar features aligned by perspective projection as guiding signals. The radar projection features are mapped to attention query vectors, and the extracted image features are mapped to key vectors and value vectors. The correlation between the two types of features is adaptively calculated through attention weights to enhance the supplementary and strengthening effect of effective radar features on image semantic features, suppress image noise and invalid background interference, and finally output the radar-guided enhanced perspective domain fusion features.

[0008] Preferably, step S3 specifically includes: The perspective enhancement features obtained in step S2 are input into the constructed depth prediction network, which predicts the depth distribution of the image scene through pixel-by-pixel regression to generate a global dense depth prediction map. Using the measured true depth of the millimeter-wave radar projection matching point as the ground truth supervision label, an L1 normalized depth supervision loss function is constructed, and the network depth prediction results are fitted to the true radar depth through backpropagation.

[0009] Preferably, step S4 specifically includes: The depth prediction map and perspective domain fusion features are introduced into the Lift-Splat spatial transformation module: first, the two-dimensional image features are up-mapped to three-dimensional geometric space through the depth prediction results, and then transformed to the bird's-eye view BEV feature space through raster aggregation projection to generate a camera-specific BEV feature map. An independent radar BEV encoding network was built to globally encode radar point clouds into radar BEV features; Finally, a channel-wise weighted stitching and fusion strategy is adopted to achieve deep fusion of camera BEV features and radar BEV features, and to construct a globally unified BEV feature representation that takes into account both image semantics and radar geometry.

[0010] Preferably, step S5 specifically includes: The first branch constructs a 3D detection head, regresses and predicts the category, 3D bounding box position, size and pose parameters of the scene target, and completes the 3D target detection task; The second branch constructs a 3D voxel occupancy prediction head, performs probability prediction on all grid voxels in 3D space, and determines whether each voxel is occupied by obstacles, vehicles or other targets, thus achieving dense occupancy modeling of the entire 3D space. The dual-branch network shares the basic features of BEV and completes collaborative parallel inference output for detection and occupation tasks.

[0011] Preferably, step S6 specifically includes: Constructing depth-occupancy geometric consistency constraints: Extract the equivalent depth corresponding to the maximum occupancy probability of each pixel position along the Z-axis of 3D spatial depth, and establish an error constraint with the dense prediction depth of the network to ensure the spatial geometric consistency of the two sets of prediction results; The global total loss function is constructed by integrating 3D detection loss, depth supervision loss, 3D occupancy loss, and consistency constraint loss, and configuring weighting coefficients. By iteratively updating all network parameters through backpropagation, multi-task end-to-end joint optimization training is achieved, ultimately resulting in a radar-camera robust 3D perception model.

[0012] Preferably, the calculation formula for the perspective domain fusion feature is:

[0013] in: This is the final cross-modal feature tensor of the perspective domain after cross-attention enhancement and fusion; This is a scaling dot product cross-attention computation function that enables adaptive allocation of feature weights; The attention query vector is obtained by linear transformation after radar 3D point perspective projection; Attention key vectors are generated by linearly mapping semantic features extracted from camera images; The attention value vector is generated by linearly mapping the semantic features extracted from the camera image.

[0014] Preferably, the formula for calculating the deep supervision loss function is:

[0015] in: is the normalized depth supervision loss value guided by the radar truth; N is the total number of valid samples that successfully match radar projection points with image pixels; (u,v) The horizontal and vertical coordinate indices of pixels in the two-dimensional image plane of the camera; For the network in pixel coordinates (u,v) The scene depth value predicted by location; For pixels (u,v) One-to-one matching of the true depth values ​​measured by radar; This is the absolute value operator, used to calculate the absolute error between the predicted depth and the true depth.

[0016] Preferably, the depth-occupancy consistency loss The calculation formula is:

[0017] in: The geometric consistency constraint loss is used for depth prediction and 3D occupancy prediction. This is a depth mapping operation that extracts the maximum occupancy probability of a single pixel along the Z-axis of 3D space. For the occupancy probability prediction tensor of voxels in the full-scene 3D space; As defined in claim 4, a pixel (u,v) The network predicts the depth value of the location; This is an L1 first-order norm operation used to quantify the overall bias between two sets of depth maps; The formula for calculating the global total loss function is as follows:

[0018] in: The global total loss function is optimized end-to-end for model training; The loss is dedicated to the 3D object detection branch; 0.5, 1.0, and 0.3 are the depth supervision losses, respectively. 3D Occupation Prediction Loss Consistency constraint loss Balanced weighted hyperparameters.

[0019] Compared with existing technologies, the advantages of this invention are: 1. This invention adopts a dual-space fusion of perspective domain and BEV domain. The perspective domain fuses radar and image features through cross-attention, while the BEV domain fuses BEV features from both types of sensors. This fully leverages radar geometric priors, achieving complementary advantages and improving the accuracy of target localization and recognition.

[0020] 2. This invention utilizes the advantage of millimeter-wave radar being unaffected by lighting and texture, using its true depth as a supervision label, and constrains depth prediction through L1 loss, which significantly reduces errors, improves the accuracy and stability of depth prediction, and provides a reliable foundation for subsequent steps.

[0021] 3. This invention constructs a depth-occupancy consistency constraint, which forces the two types of prediction results to be geometrically consistent, mutually correct and supplement each other, effectively solves cross-task contradictions, enhances the ability to model occluded areas, and improves the completeness and accuracy of perception. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0023] Figure 2 This is a flowchart of the image multi-scale feature extraction and radar 3D point perspective projection alignment part of the present invention.

[0024] Figure 3 This is a flowchart of the perspective feature cross-space conversion and dual-modal BEV feature fusion part in this invention.

[0025] Figure 4 This is a flowchart of the joint optimization part of depth-occupancy geometric consistency constraint and total loss in this invention. Detailed Implementation

[0026] To facilitate understanding of this application and to make the aforementioned objectives, features, and advantages of this application more apparent, a detailed description of specific embodiments of this application is provided below in conjunction with the accompanying drawings. Numerous specific details are set forth in the following description to provide a thorough understanding of this application.

[0027] Reference Figures 1-4 As shown, the 3D target perception method based on dual-space fusion and depth-occupancy consistency is characterized by the following steps: Step 1: Image multi-scale feature extraction and radar 3D point perspective projection alignment (1) Multi-scale feature extraction of images: A pre-trained deep convolutional backbone network (selected from ResNet50 and EfficientNet-B4) is used to extract multi-scale features from the input original RGB images of the camera. The shallow texture features, mid-level semantic features and deep global features are output layer by layer to form a high-dimensional dense image feature tensor, which provides semantic support for subsequent cross-modal fusion. (2) Radar 3D point preprocessing and coordinate transformation: Based on the pre-calibrated intrinsic parameter matrix (including camera focal length and pixel size) and extrinsic parameter transformation matrix (including the relative position and attitude of radar and camera), the coordinate transformation from radar coordinate system to camera coordinate system is completed first, and noise points and abnormal points in radar 3D point cloud are removed, while valid radar 3D points are retained. (3) Perspective projection and alignment: Through the perspective projection geometric model, all radar 3D points after coordinate transformation and preprocessing are mapped to the two-dimensional image plane of the camera to achieve precise spatial alignment between radar point cloud and image pixels; at the same time, the radar points after projection are feature-encoded to generate radar perspective projection features, ensuring the spatial consistency between radar perspective projection features and image multi-scale features in the pixel dimension, laying the foundation for subsequent cross-modal feature fusion.

[0028] Step 2: Perspective-domain radar-camera cross-attention feature fusion Based on the spatial alignment results achieved in step 1, a radar-camera cross-modal attention fusion module is constructed in the perspective domain to achieve deep fusion of features from the two modalities. Specifically, the features projected from the radar 3D points are transformed linearly to generate an attention query vector. The image features extracted in step 1 are linearly mapped to generate attention key vectors. Sum value vector The similarity between the query vector and the key vector is calculated through a cross-attention operation module to obtain attention weights and perform normalization. The contribution of the two types of features is adaptively allocated to enhance the supplementation of radar geometric features to image semantic features, suppress image noise and invalid background interference, and finally output the perspective domain cross-modal fusion feature tensor. The core calculation formula is:

[0029] The variables are explained below: This is the final cross-modal feature tensor of the perspective domain after cross-attention enhancement and fusion; This is a scaling dot product cross-attention computation function that enables adaptive allocation of feature weights; The attention query vector is obtained by linear transformation after radar 3D point perspective projection; Attention key vectors are generated by linearly mapping semantic features extracted from camera images; The attention value vector is generated by linearly mapping the semantic features extracted from the camera image.

[0030] Step 3: Construction of Deep Prediction Network and Deep Supervision Constraints for Radar Ground Value (1) Construction of deep prediction network: The deep prediction network consists of convolutional layers and deconvolutional layers, which are used to fuse features in the perspective domain. As input, pixel-wise depth regression is used to output a globally dense image depth prediction map. ,in Represents the two-dimensional pixel coordinates of an image (u,v) The network prediction depth at the location, in meters; (2) Radar ground truth depth acquisition: Select the radar point depth that successfully matches the image pixel in step 2 as the ground truth label, and obtain the depth of the pixel. (u,v) One-to-one matching of radar measured true depth values ; (3) Radar truth depth supervision constraint: Constructing an L1 normalized depth supervision loss function By constraining the output of the depth prediction network through backpropagation, the model is guided to fit the true radar depth, correcting depth prediction biases in textureless and low-light areas of the image, and improving the accuracy and stability of monocular depth estimation. The calculation formula is as follows:

[0031] The variables are explained below: is the normalized depth supervision loss value guided by the radar truth; N is the total number of valid samples that successfully match radar projection points with image pixels; (u,v) The horizontal and vertical coordinate indices of pixels in the two-dimensional image plane of the camera; For the network at pixel coordinates (u,v) The scene depth value predicted by location; For pixels (u,v) One-to-one matching of the true depth values ​​measured by radar; This is the absolute value operator, used to calculate the absolute error between the predicted depth and the true depth.

[0032] Step 4: Perspective feature cross-space transformation and dual-modal BEV feature fusion (1) Perspective feature cross-space transformation: Introduce the Lift-Splat spatial transformation module, using the depth prediction map output in step 3. Features of fusion with perspective domain As input, the 2D image features are first up-mapped to a 3D geometric space using depth prediction results. Then, through raster aggregation and projection transformation, they are transformed into a bird's-eye view (BEV) space, generating camera BEV features that combine semantic and depth information. ; (2) BEV domain radar feature encoding: An independent radar BEV encoding network is built to globally encode the radar 3D point cloud preprocessed in claim 2, and output radar BEV features containing radar geometric priors. ; (3) Dual-modal BEV feature fusion: A channel-by-channel weighted stitching fusion strategy is adopted to fuse camera BEV features. Radar BEV characteristics Deep integration is performed to construct a globally unified BEV fusion feature tensor. It takes into account both image semantic details and radar geometric accuracy, providing a unified feature input for subsequent 3D perception tasks.

[0033] Step 5: Joint prediction of 3D detection and 3D occupancy under BEV fusion features Based on the globally fused BEV features output in step 4, a dual-branch parallel prediction network structure is constructed: (1) Construction of dual-branch network: The joint prediction network includes a 3D detection branch and a 3D occupancy prediction branch. The two branches share the BEV fusion feature tensor. Each of them completes its corresponding sensory task; (2) 3D detection branch prediction: The 3D detection branch is constructed through convolutional and fully connected layers, in order to... As input, the regression predicts the category and 3D bounding box location of the target in the scene. (x,y,z) The system uses size and attitude parameters to perform discrete target detection tasks, including coordinates. (x,y,z) The unit is meters, with the BEV domain origin as the reference. (3) 3D Occupation Prediction Branch Prediction: The 3D occupancy prediction branch divides the 3D space into voxel grids, with... As input, predict the occupancy probability of each voxel (ranging from 0 to 1), and output a 3D voxel occupancy prediction tensor. It determines whether a voxel is occupied by a target and completes dense space modeling for the entire scene; dual-branch collaborative parallel reasoning improves prediction efficiency and consistency.

[0034] Step 6: Joint optimization of depth-occupancy geometric consistency constraints and total loss Establish depth-occupancy geometric consistency constraints, integrate the losses from various tasks to construct a total loss function, and achieve joint model optimization, specifically including: (1) Depth-occupancy geometric consistency constraint: along the three-dimensional spatial depth Z-axis, extract the equivalent depth corresponding to the maximum occupancy probability of each pixel position, and compare it with the depth prediction map output by claim 4. An L1 norm constraint is established to force the two types of predictions to maintain spatial geometric consistency, using depth-occupancy consistency loss. The calculation formula is:

[0035] The variables are explained below: The geometric consistency constraint loss is used for depth prediction and 3D occupancy prediction. This is a depth mapping operation that extracts the maximum occupancy probability of a single pixel along the Z-axis of 3D space. For the occupancy probability prediction tensor of voxels in the full-scene 3D space; As defined in claim 4, a pixel (u,v) The network predicts the depth value of the location; This is an L1 first-order norm operation used to quantify the overall bias between two sets of depth maps; (2) Calculation of each sub-loss: ① 3D target detection loss The weighted sum of classification loss and bounding box regression loss is used, where the classification loss is the cross-entropy loss and the bounding box regression loss is the smoothL1 loss. ② Radar depth surveillance loss As described in claim 4, the calculation formula is as follows: ; ③ 3D voxel occupancy prediction loss The cross-entropy loss function is used to constrain the accuracy of the 3D voxel occupancy prediction. ④ Depth-occupancy geometric consistency loss As defined in step (1) above, the calculation formula is: ; (3) Joint optimization of total loss: The above sub-losses are combined according to preset weights to construct the total loss function. The core calculation formula is:

[0036] The variables are explained below: The global total loss function is optimized end-to-end for model training; The loss is dedicated to the 3D object detection branch; 0.5, 1.0, and 0.3 are the depth supervision losses, respectively. 3D Occupation Prediction Loss Consistency constraint loss The model employs a balanced weighted hyperparameter; during model training, all network parameters are iteratively updated through backpropagation until the total loss converges, achieving end-to-end joint optimization across multiple tasks, ultimately resulting in a high-precision and robust 3D perception model.

[0037] This invention addresses the shortcomings of existing radar-camera fusion 3D perception technologies, such as limited feature fusion methods, insufficient depth prediction accuracy, spatial inconsistencies in multi-task prediction, and poor adaptability to harsh scenes. It constructs a dual-space collaborative fusion architecture of perspective domain and BEV domain, achieving depth enhancement of radar and image features in the perspective domain through cross-attention. A supervised loss is constructed using the true depth of millimeter-wave radar, effectively improving depth estimation accuracy in distant and weakly textured regions. Simultaneously, a depth-occupancy geometric consistency constraint is added to resolve cross-task prediction conflicts and enhance the ability to model occluded regions. A dual-branch network is built based on shared BEV fusion features to achieve end-to-end joint prediction of 3D detection and 3D occupancy. Combined with a multi-loss weighted joint optimization strategy, this comprehensively enhances the model's perception stability and overall robustness under complex conditions such as low light, rain, fog, and nighttime.

[0038] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A 3D target perception method based on dual-space fusion and depth-occupancy consistency, characterized in that, Includes the following steps: S1. Acquire multi-view camera images and millimeter-wave radar point clouds, and perform multi-scale feature extraction of images and radar 3D point perspective projection alignment respectively. S2. Construct a cross-modal cross-attention fusion module in the perspective domain, using radar projection features as query vectors and image features as key vectors, adaptively fusing the two types of features, and outputting the enhanced perspective domain fusion features. S3. Construct a depth prediction network based on perspective domain fusion, and combine it with radar ground truth depth to achieve supervision and constraint. S4. Perform cross-space transformation on the optimized perspective domain fusion features, and simultaneously encode the radar 3D point cloud in the BEV domain to achieve dual-modal BEV feature fusion. S5. A dual-branch parallel prediction network is built based on the BEV fusion feature tensor to achieve the joint output of 3D detection and 3D occupancy. S6. Establish depth-occupancy geometric consistency constraints, integrate the losses of each task to construct the total loss function, and achieve joint optimization of the model.

2. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 1, characterized in that: Step S1 specifically includes: We collect raw RGB image data from multi-view cameras, and use a deep convolutional backbone network to extract multi-level and multi-scale semantic features from the images. We mine the shallow texture contour features, mid-level target semantic features and deep global scene features of the images layer by layer, and output a high-dimensional dense image feature tensor. Acquire three-dimensional spatial point cloud data collected by millimeter-wave radar, and complete the coordinate transformation from radar coordinate system to camera coordinate system based on the pre-calibrated intrinsic parameter matrix and extrinsic parameter transformation matrix of radar and camera; By using a perspective projection geometry model, all three-dimensional spatial points of the radar are mapped onto the two-dimensional pixel plane of the camera, achieving precise alignment and matching between the radar point cloud and image pixels in spatial dimensions.

3. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 2, characterized in that: Step S2 specifically includes: A cross-modal cross-attention fusion mechanism is constructed using radar features aligned by perspective projection as guiding signals. The radar projection features are mapped to attention query vectors, and the extracted image features are mapped to key vectors and value vectors. The correlation between the two types of features is adaptively calculated through attention weights to enhance the supplementary and strengthening effect of effective radar features on image semantic features, suppress image noise and invalid background interference, and finally output the radar-guided enhanced perspective domain fusion features.

4. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 3, characterized in that: Step S3 specifically includes: The perspective enhancement features obtained in step S2 are input into the constructed depth prediction network, which predicts the depth distribution of the image scene through pixel-by-pixel regression to generate a global dense depth prediction map. Using the measured true depth of the millimeter-wave radar projection matching point as the ground truth supervision label, an L1 normalized depth supervision loss function is constructed, and the network depth prediction results are fitted to the true radar depth through backpropagation.

5. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 4, characterized in that: Step S4 specifically includes: The depth prediction map and perspective domain fusion features are introduced into the Lift-Splat spatial transformation module: first, the two-dimensional image features are up-mapped to three-dimensional geometric space through the depth prediction results, and then transformed to the bird's-eye view BEV feature space through raster aggregation projection to generate a camera-specific BEV feature map. An independent radar BEV encoding network was built to globally encode radar point clouds into radar BEV features; Finally, a channel-wise weighted stitching and fusion strategy is adopted to achieve deep fusion of camera BEV features and radar BEV features, and to construct a globally unified BEV feature representation that takes into account both image semantics and radar geometry.

6. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 5, characterized in that: Step S5 specifically includes: The first branch constructs a 3D detection head, regresses and predicts the category, 3D bounding box position, size and pose parameters of the scene target, and completes the 3D target detection task; The second branch constructs a 3D voxel occupancy prediction head, performs probability prediction on all grid voxels in 3D space, and determines whether each voxel is occupied by obstacles, vehicles or other targets, thus achieving dense occupancy modeling of the entire 3D space. The dual-branch network shares the basic features of BEV and completes collaborative parallel inference output for detection and occupation tasks.

7. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 6, characterized in that: Step S6 specifically includes: Constructing depth-occupancy geometric consistency constraints: Extract the equivalent depth corresponding to the maximum occupancy probability of each pixel position along the Z-axis of 3D spatial depth, and establish an error constraint with the dense prediction depth of the network to ensure the spatial geometric consistency of the two sets of prediction results; The global total loss function is constructed by integrating 3D detection loss, depth supervision loss, 3D occupancy loss, and consistency constraint loss, and configuring weighting coefficients. By iteratively updating all network parameters through backpropagation, multi-task end-to-end joint optimization training is achieved, ultimately resulting in a radar-camera robust 3D perception model.

8. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 2, characterized in that: The calculation formula for the perspective domain fusion feature is as follows: in: This is the final cross-modal feature tensor of the perspective domain after cross-attention enhancement and fusion; This is a scaling dot product cross-attention computation function that enables adaptive allocation of feature weights; The attention query vector is obtained by linear transformation after radar 3D point perspective projection; Attention key vectors are generated by linearly mapping semantic features extracted from camera images; The attention value vector is generated by linearly mapping the semantic features extracted from the camera image.

9. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 3, characterized in that: The formula for calculating the deep supervision loss function is as follows: in: is the normalized depth supervision loss value guided by the radar truth; N is the total number of valid samples that successfully match radar projection points with image pixels; (u,v) The horizontal and vertical coordinate indices of pixels in the two-dimensional image plane of the camera; For the network in pixel coordinates (u,v) The scene depth value predicted by location; For pixels (u,v) One-to-one matching of the true depth values ​​measured by radar; This is the absolute value operator, used to calculate the absolute error between the predicted depth and the true depth.

10. The 3D target perception method based on dual-space fusion and depth-occupancy consistency according to claim 7, characterized in that: The depth-occupancy consistency loss The calculation formula is: in: The geometric consistency constraint loss is used for depth prediction and 3D occupancy prediction. This is a depth mapping operation that extracts the maximum occupancy probability of a single pixel along the Z-axis of 3D space. For the occupancy probability prediction tensor of voxels in the full-scene 3D space; As defined in claim 4, a pixel (u,v) The network predicts the depth value of the location; This is an L1 first-order norm operation used to quantify the overall bias between two sets of depth maps; The formula for calculating the global total loss function is as follows: in: The global total loss function is optimized end-to-end for model training; The loss is dedicated to the 3D object detection branch; 0.5, 1.0, and 0.3 are the depth supervision losses, respectively. 3D Occupation Prediction Loss Consistency constraint loss Balanced weighted hyperparameters.

Images