4D millimeter wave radar 3D target detection method based on multi-scale feature fusion

Abstract

This invention provides a 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion, comprising: inputting point cloud data; aggregating features of points within a column using a feature encoder, based on geometric distribution and attention methods, to obtain features for each column; inputting bird's-eye view features into a 2D backbone network for further feature extraction. The bird's-eye view features are input into a three-layer top-down convolutional network to obtain three feature maps at different scales; the three feature maps at different scales are respectively input into a three-channel convolutional network, while simultaneously introducing intermediate features from other channels into the corresponding channels for fusion, resulting in a highly aggregated feature map; upsampling the obtained feature map to restore resolution, and simultaneously introducing the three feature maps obtained from the 2D backbone network for fusion, generating a feature map with high representational capability; and obtaining the detection result through a detection head. This invention can improve detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of point cloud 3D target detection methods, specifically to a 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion. Background Technology

[0002] 3D object detection is one of the most popular and challenging tasks in 3D perception for autonomous driving systems. It aims to predict the position, size, and category of key objects in 3D space, such as cars, pedestrians, and cyclists, focusing on the localization and identification of objects in a real-world three-dimensional coordinate system. The geometric information predicted by 3D object detection can be directly used to measure the distance between the autonomous vehicle and key targets, thereby helping to plan driving routes and avoid collisions.

[0003] The rapid development of millimeter-wave radar technology in recent years has led to the emergence of 4D millimeter-wave radar. 4D millimeter-wave radar can provide coordinate and velocity information, exhibits sufficient stability under extreme weather conditions, and is relatively inexpensive. Furthermore, it has the ability to measure elevation angles, meaning that 4D millimeter-wave radar can capture altitude information. This additional altitude information significantly improves the accuracy of target detection and localization.

[0004] 4D millimeter-wave radar provides point cloud data similar to lidar, so lidar-based 3D target detection methods can be easily applied to 4D millimeter-wave radar. However, due to the limited angular resolution and multipath propagation of radar signals, 4D millimeter-wave radar point clouds contain many noisy points and are much sparser than lidar point clouds. This means that 4D millimeter-wave radar point clouds provide less geometric and semantic information. Directly applying lidar-based methods to 4D millimeter-wave radar may produce poor results. Furthermore, how to fully utilize the velocity information provided by 4D millimeter-wave radar is also a current challenge. Currently, 4D millimeter-wave radar methods are still underdeveloped, and there is an urgent need to develop more specialized, efficient, and accurate 4D millimeter-wave radar-based methods.

[0005] Methods for 3D object detection in point clouds are generally divided into point-based methods and voxel-based methods. Point-based methods, derived from the PointNet network model, utilize raw points from an irregular 3D point cloud space to extract features. This method is based on the assumption that neighboring points within a certain range can cluster into a single point, directly extracting point-by-point features. The subsequent PointNet++, as an enhanced version, integrates the ability to extract local features, capturing fine geometric structures from small neighborhoods. However, point-based methods always face significant computational challenges, making them unsuitable for detection tasks involving large point cloud environments. Voxel-based methods, derived from the groundbreaking VoxelNet network model, divide the irregular point cloud into a pre-defined voxel grid of equal size. By introducing a voxel feature encoder, a set of points within each voxel is converted into a unified feature representation. The subsequent SECOND network model introduces 3D sparse convolution to process the voxel grid, reducing the computation of empty voxels and significantly improving training and inference speed. Furthermore, the PointPillars network model uses vertical pillars as a special voxel representation, extracts intra-pillar point cloud features using the PointNet method, and processes these features with a 2D backbone network, thus significantly improving detection speed and accuracy. However, these methods are all based on LiDAR point clouds and cannot be well adapted to the high noise and high sparsity of 4D millimeter-wave radar, nor do they utilize the velocity information of 4D millimeter-wave radar. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion. Utilizing a 4D millimeter-wave radar feature encoder, the method fully integrates point coordinates, velocity, and reflectivity information within a cylinder pixel partition, improving point representation capabilities and mitigating the impact of high noise and high sparsity in 4D millimeter-wave radar point clouds. Furthermore, a multi-scale feature interaction module is used. After feature extraction from a 2D backbone network, the output feature maps at different scales are deeply fused through a top-down convolutional network, and then restored to the same scale by a bottom-up upsampling network. This fusion and interaction processing of the extracted feature maps at different scales enriches the information in the feature maps, improves detection accuracy, and enhances the representation capabilities of the feature maps.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion includes the following steps:

[0009] STEP 1: Input point cloud data N×d; where N is the number of points and d is the input feature dimension of the point cloud, including three-dimensional coordinates, velocity information and reflectivity;

[0010] STEP2: Divide the point cloud space into vertical columns of the same size according to the preset dimensions;

[0011] STEP3: The radar feature encoder aggregates the features of the points within the column according to the geometric distribution and attention method to obtain the features of each column;

[0012] STEP4: Based on the positional information of the columns, convert the column features into bird's-eye view features;

[0013] STEP 5: Input the bird's-eye view features into a three-layer top-down convolutional network to obtain three feature maps at different scales;

[0014] STEP 6: Input the three feature maps at different scales into a three-channel convolutional network, and simultaneously introduce intermediate features from other channels into the corresponding channels for fusion to obtain a highly aggregated feature map:

[0015] STEP 7: Design a multi-scale feature interaction module. In the multi-scale feature interaction module, feature maps of three different scales are input into a three-channel convolutional network. At the same time, intermediate features from other channels are introduced into the corresponding channels for fusion to obtain a highly aggregated feature map.

[0016] STEP8: Obtain the detection results through the detection head.

[0017] Furthermore, the radar feature encoder of the STEP2 consists of a column feature encoder and a raw feature fusion encoder.

[0018] Furthermore, the point cloud space is divided into vertical cylinders of equal size, resulting in... There are 3 non-empty columns, and each non-empty column is randomly sampled. fewer than 1 point Each point is filled with zero-value points, and each point is represented by its spatial coordinates, reflectivity, relative radial velocity, and radial compensation velocity as follows:

[0019] F =[ x , y , z , rcs , v r , v com ] ;

[0020] in, For each point, the feature representation, Let be the coordinates of the point in the point cloud space. For reflectivity, These are the relative radial velocity and the radial compensation velocity of the point, respectively.

[0021] The segmented point cloud data is represented as tensor;

[0022] For a column feature encoder, the deviation of each point from the mean of all points within the corresponding column and the coordinate offset from the geometric center of the column are calculated as an extension of the original feature, expressed as:

[0023] F ext =[ x o , y o , z o , rcs o , v r o , v co m o , x c , y c , z c ] ;

[0024] in, For extended features of points, [ x o , y o , z o , rcs o , v r o , v co m o ] The deviations of the point's position coordinates, reflectivity, relative radial velocity, and radial compensation velocity from the mean of all points within the cylinder. [ x c , y c , z c ] The coordinate offset of the point with respect to the geometric center of the cylinder;

[0025] The extended column feature is represented as tensor , is represented as:

[0026] F m =[ x , y , z , rcs , v r , v com , x o , y o , z o , rcs o , v r o , v co m o , x c , y c , z c ] ;

[0027] Then, the features are processed sequentially using linear layers, batch normalization layers, activation functions, and max pooling layers to obtain... cylindrical feature tensor Specifically:

[0028] ;

[0029] in, For max pooling operation, For activation function, This is a batch normalization operation.

[0030] Furthermore, in the original feature fusion encoder, the original point features are decomposed into coordinates. Reflectivity and speed , is represented as:

[0031] F coor =[ x , y , z ] ;

[0032] ;

[0033] F v =[ v r , v com ] ;

[0034] for , , The importance of each dimension is calculated using a linear layer and a normalized exponential function, and then multiplied with the original features to obtain secondary features. , is represented as:

[0035] ;

[0036] ;

[0037] F sec =[ F coor sec , F rcs sec , F velo sec ] ;

[0038] in, To calculate the importance weights of secondary features, For normalized exponential functions, This is a linear layer operation; the superscript T indicates the transpose of the matrix; for , , One of them, for The corresponding secondary features, i.e. , , One of them, for , , One of them, , , Represents coordinates, reflectivity, and velocity;

[0039] The obtained secondary feature groups are divided into multiple secondary feature groups, represented as follows:

[0040] ;

[0041] F coor & velo sec = [ F coor sec , F velo sec ] ;

[0042] ;

[0043] ;

[0044] in, Secondary feature set representing coordinates and reflectivity. Secondary feature sets representing coordinates and velocities. Secondary feature groups representing reflectivity and velocity, A secondary feature set representing the overall coordinates, reflectivity, and velocity;

[0045] The above-mentioned secondary feature groups are processed in the same way as the original feature groups to obtain tertiary features. Therefore, the final fusion feature is represented as tensor :

[0046] F RFF =[ F sec , F thi ] ;

[0047] Then, the features are processed sequentially using linear layers, batch normalization layers, activation functions, and max pooling layers to obtain... Fusion feature tensor ;

[0048] Finally, the column feature tensor and fusion feature tensor exist Dimensional splicing yields tensor Tensors are divided according to their columns. Disperse back to the original position, and obtain Bird's-eye view features, and Indicates the height and width of the feature.

[0049] Furthermore, in STEP7, the multi-scale feature interaction module consists of a top-down convolutional network and a bottom-up upsampling network.

[0050] Furthermore, the structure of the top-down convolutional network is as follows:

[0051] The three different scales of input feature maps from the 2D backbone network are respectively of , of , of ,in, For the number of channels, and Indicates the height and width of the feature;

[0052] These feature maps are convolved, and a fusion operation of these feature maps at different scales is introduced. Each convolution operation is followed by a batch normalization layer and an activation function. Finally, a feature map that integrates all features is obtained. Specifically:

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] in, It is the feature map after the input feature map has undergone convolutional preprocessing. for The feature map obtained after the first fusion process, for The feature map obtained after the first fusion process, for The feature map obtained after the second fusion process, For activation function, For batch normalization operations, This is a convolution operation.

[0059] Furthermore, the structure of the bottom-up upsampling network is as follows:

[0060] Feature map that integrates all features Upsampled to size Feature map Subsequently, in the feature map Feature maps are introduced sequentially during the upsampling operation. and feature map that integrates all features The fusion yields a size of Feature map Simultaneously, three feature maps from the 2D backbone network were upsampled to the same level as the feature maps. The feature maps are of the same size; finally, the above feature maps are concatenated along the channel dimension to obtain the final output feature. Specifically:

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] ;

[0068] in, for The feature map obtained after the first upsampling process, for The feature map obtained after the first fusion upsampling process, for The feature map obtained after the first fusion upsampling process, for The feature map obtained after the upsampling operation for The feature map obtained after the upsampling operation for The feature map obtained after the upsampling operation For upsampling operation, This is for splicing operations.

[0069] Beneficial effects:

[0070] 1. This invention proposes a radar original feature fusion encoder, which increases the utilization rate of original information (spatial coordinates, velocity, reflectivity) of 4D millimeter wave radar point cloud.

[0071] 2. To address the issues of high sparsity and high noise in 4D millimeter-wave radar point clouds, this invention proposes a multi-scale feature interaction module that fuses feature maps of different resolutions during the convolution and upsampling stages, thereby improving the representational capability of feature maps at different scales. Attached Figure Description

[0072] Figure 1 This is a flowchart of a 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion according to the present invention;

[0073] Figure 2 This is a diagram of the radar feature encoder network structure.

[0074] Figure 3 This is a diagram of the original feature fusion encoder network structure.

[0075] Figure 4 This is a diagram of a top-down convolutional network structure.

[0076] Figure 5 This is a diagram of a bottom-up upsampling network structure.

[0077] Figure 6 This is a diagram showing the detection effect of the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0079] Figure 1 As shown, the 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion of the present invention includes the following steps:

[0080] STEP1: Input point cloud data N×d; where N is the number of points and d is the input feature dimension of the point cloud, which generally includes three-dimensional coordinates, velocity information and reflectivity.

[0081] STEP2: In order to extract point cloud information, the point cloud space is divided into vertical columns of the same size according to the pre-set dimensions;

[0082] STEP3: The radar feature encoder aggregates the features of the points within the column according to the geometric distribution and attention methods to obtain the features of each column;

[0083] STEP4: Based on the positional information of the columns, convert the column features into bird's-eye view features;

[0084] STEP 5: Input the bird's-eye view features into a three-layer top-down convolutional network to obtain three feature maps at different scales;

[0085] STEP 6: Input the three feature maps at different scales into a three-channel convolutional network, and simultaneously introduce intermediate features from other channels into the corresponding channels for fusion to obtain a highly aggregated feature map.

[0086] STEP 7: Upsample the obtained feature map to restore resolution. Simultaneously, fuse the three feature maps obtained from the 2D backbone network to generate a feature map with high representational power, including:

[0087] To alleviate the information loss caused by the sparsity of 4D millimeter-wave radar point clouds and improve the representational ability of feature maps, this invention proposes a multi-scale feature interaction module. In this module, feature maps of three different scales are input into a three-channel convolutional network, and intermediate features from other channels are introduced into the corresponding channels for fusion to obtain a highly aggregated feature map.

[0088] STEP8: Obtain the detection results through the detection head.

[0089] Specifically, in STEP2, to fully utilize the original features of the 4D millimeter-wave radar point cloud, this invention proposes a radar feature encoder. This radar feature encoder consists of a conventional cylindrical feature encoder and an original feature fusion encoder, with the overall structure as follows: Figure 2 As shown.

[0090] By dividing the point cloud space into vertical cylinders of equal size (i.e., vertical cylinderization), we obtain... There are 3 non-empty columns, and each non-empty column is randomly sampled. fewer than 100 points Each point is filled with zero-value points, and each point is represented by its spatial coordinates, reflectivity, relative radial velocity, and radial compensation velocity as follows:

[0091] F =[ x , y , z , rcs , v r , v com ] ;

[0092] in, For each point, the feature representation, Let be the coordinates of the point in the point cloud space. For reflectivity, These are the relative radial velocity and the radial compensation velocity of the point, respectively.

[0093] The segmented point cloud data can be represented as The tensor.

[0094] For a conventional column feature encoder, this invention calculates the deviation of each point from the mean of all points within the corresponding column and the coordinate offset about the geometric center of the column as an extension of the original feature, expressed as:

[0095] F ext =[ x o , y o , z o , rcs o , v r o , v co m o , x c , y c , z c ] ;

[0096] in, For extended features of points, [ x o , y o , z o , rcs o , v r o , v co m o ] The deviations of the point's position coordinates, reflectivity, relative radial velocity, and radial compensation velocity from the mean of all points within the cylinder. [ x c , y c , z c ] The coordinate offset of the point with respect to the geometric center of the cylinder;

[0097] The extended column feature can be represented as tensor , is represented as:

[0098] F m =[ x , y , z , rcs , v r , v com , x o , y o , z o , rcs o , v r o , v co m o , x c , y c , z c ] ;

[0099] Then, the features are processed sequentially using linear layers, batch normalization layers, activation functions, and max pooling layers to obtain... cylindrical feature tensor Specifically:

[0100] ;

[0101] in, For max pooling operation, For activation function, This is a batch normalization operation.

[0102] The network structure of the original feature fusion encoder is as follows: Figure 3 As shown.

[0103] This invention decomposes the original features of a point into coordinates. Reflectivity and speed , is represented as:

[0104] F coor =[ x , y , z ] ;

[0105] ;

[0106] F v =[ v r , v com ] ;

[0107] Each of the above features , , This invention utilizes a linear layer and a normalized exponential function to calculate the importance of each dimension, and multiplies it with the original features to obtain secondary features. , is represented as:

[0108] ;

[0109] ;

[0110] F sec =[ F coor sec , F rcs sec , F velo sec ] ;

[0111] in, To calculate the importance weights of secondary features, For normalized exponential functions, This is a linear layer operation. The superscript T denotes the transpose of the matrix. for , , One of them, for The corresponding secondary features, i.e. , , One of them, for , , One of them, , , Represents coordinates, reflectivity, and velocity.

[0112] The obtained secondary features are grouped into multiple secondary feature groups, as follows:

[0113] ;

[0114] F coor & velo sec = [ F coor sec , F velo sec ] ;

[0115] ;

[0116] ;

[0117] in, Secondary feature set representing coordinates and reflectivity. Secondary feature sets representing coordinates and velocities. Secondary feature groups representing reflectivity and velocity, A secondary feature group representing the overall coordinates, reflectivity, and velocity.

[0118] The above-mentioned secondary feature groups are processed in the same way as the original feature groups to obtain tertiary features. Therefore, the final fusion feature can be expressed as tensor :

[0119] F RFF =[ F sec , F thi ] ;

[0120] Then, the features are processed sequentially using linear layers, batch normalization layers, activation functions, and max pooling layers to obtain... Fusion feature tensor .

[0121] Finally, the column feature tensor and fusion feature tensor exist Dimensional splicing yields tensor Tensors are divided according to their columns. Disperse back to the original position, and obtain Bird's-eye view features, and Indicates the height and width of the feature.

[0122] Specifically, STEP7 includes:

[0123] In a 2D backbone network, three feature maps of different scales and resolutions are output. This invention proposes a multi-scale feature interaction module to fuse feature maps of different scales. The entire multi-scale feature interaction module consists of a top-down convolutional network and a bottom-up upsampling network.

[0124] Top-down convolutional network structure as follows: Figure 4 As shown.

[0125] The three different scales of input feature maps from the 2D backbone network are respectively of , of , of ,in, For the number of channels, and Indicates the height and width of the feature.

[0126] This invention performs convolutions on these feature maps and introduces a fusion operation on these feature maps at different scales. Each convolution operation is followed by a batch normalization layer and an activation function. Finally, a feature map that integrates all features is obtained. Specifically:

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] ;

[0132] in, It is the feature map after the input feature map has undergone convolutional preprocessing. for The feature map obtained after the first fusion process, for The feature map obtained after the first fusion process, for The feature map obtained after the second fusion process, For activation function, For batch normalization operations, This is a convolution operation.

[0133] Bottom-up upsampling network structure such as Figure 5 As shown.

[0134] This invention integrates feature maps containing all features. Upsampled to size Feature map Subsequently, in the feature map Feature maps are introduced sequentially during the upsampling operation. and feature map that integrates all features The fusion yields a size of Feature map Simultaneously, three feature maps from the 2D backbone network were upsampled to the same level as the feature maps. The feature maps are of the same size. Finally, the above feature maps are concatenated along the channel dimension to obtain the final output feature. Specifically:

[0135] ;

[0136] ;

[0137] ;

[0138] ;

[0139] ;

[0140] ;

[0141] ;

[0142] in, for The feature map obtained after the first upsampling process, for The feature map obtained after the first fusion upsampling process, for The feature map obtained after the first fusion upsampling process, for The feature map obtained after the upsampling operation for The feature map obtained after the upsampling operation for The feature map obtained after the upsampling operation For upsampling operation, This is for splicing operations.

[0143] like Figure 6 As shown, this invention uses the Outdoor 4D Millimeter Wave Radar Point Cloud Dataset (VoD) proposed by Delft University. It can be seen that the method of this invention has excellent performance. The upper layer is the image scene, and the lower layer is the corresponding point cloud scene. The light gray box is the detected target box, and the white box is the real target box.

Claims

1. A 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion, characterized in that, Includes the following steps: STEP 1: Input point cloud data N×d; where N is the number of points and d is the input feature dimension of the point cloud, including three-dimensional coordinates, velocity information and reflectivity; STEP2: Divide the point cloud space into vertical columns of the same size according to a pre-set dimension; STEP2 uses a radar feature encoder, which consists of a column feature encoder and a raw feature fusion encoder; In the original feature fusion encoder, the original point features are split into coordinates. Reflectivity and speed , is represented as: ； ； ； in, Let be the coordinates of the point in the point cloud space. For reflectivity, These are the relative radial velocity and the radial compensation velocity of the point, respectively. for , , The importance of each dimension is calculated using a linear layer and a normalized exponential function, and then multiplied with the original features to obtain secondary features. , is represented as: ； ； ； in, To calculate the importance weights of secondary features, For normalized exponential functions, This is a linear layer operation; the superscript T indicates the transpose of the matrix; for , , One of them, for The corresponding secondary features, i.e. , , One of them, for , , One of them, , , Represents coordinates, reflectivity, and velocity; The obtained secondary feature groups are divided into multiple secondary feature groups, represented as follows: ； ； ； ； in, Secondary feature set representing coordinates and reflectivity. Secondary feature sets representing coordinates and velocities. Secondary feature groups representing reflectivity and velocity, A secondary feature set representing the overall coordinates, reflectivity, and velocity; The above-mentioned secondary feature groups are processed in the same way as the original feature groups to obtain tertiary features. Therefore, the final fusion feature is represented as tensor : ； Then, the features are processed sequentially using linear layers, batch normalization layers, activation functions, and max pooling layers to obtain... Fusion feature tensor P represents the number of non-empty columns; Finally, the column feature tensor and fusion feature tensor exist Dimensional splicing yields tensor Tensors are divided according to their columns. Disperse back to the original position, and obtain Bird's-eye view features, and Indicates the height and width of the feature; STEP3: The radar feature encoder aggregates the features of the points within the column according to the geometric distribution and attention method to obtain the features of each column; STEP4: Based on the positional information of the columns, convert the column features into bird's-eye view features; STEP 5: Input the bird's-eye view features into a three-layer top-down convolutional network to obtain three feature maps at different scales; STEP 6: Input the three feature maps at different scales into a three-channel convolutional network, and simultaneously introduce intermediate features from other channels into the corresponding channels for fusion to obtain a highly aggregated feature map: STEP 7: Design a multi-scale feature interaction module. In the multi-scale feature interaction module, feature maps of three different scales are input into a three-channel convolutional network. At the same time, intermediate features from other channels are introduced into the corresponding channels for fusion to obtain a highly aggregated feature map. STEP8: Obtain the detection results through the detection head.

2. The 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion according to claim 1, characterized in that, Dividing the point cloud space into vertical cylinders of equal size, we obtain... There are 3 non-empty columns, and each non-empty column is randomly sampled. fewer than 1 point Each point is filled with zero-value points, and each point is represented by its spatial coordinates, reflectivity, relative radial velocity, and radial compensation velocity as follows: ； in, Feature representation for each point; The segmented point cloud data is represented as tensor; For a column feature encoder, the deviation of each point from the mean of all points within the corresponding column and the coordinate offset from the geometric center of the column are calculated as an extension of the original feature, expressed as: ； in, For extended features of points, The deviations of the point's position coordinates, reflectivity, relative radial velocity, and radial compensation velocity from the mean of all points within the cylinder. The coordinate offset of the point with respect to the geometric center of the cylinder; The extended column feature is represented as tensor , is represented as: ； Then, the features are processed sequentially using linear layers, batch normalization layers, activation functions, and max pooling layers to obtain... cylindrical feature tensor Specifically: ； in, For max pooling operation, For activation function, This is a batch normalization operation.

3. The 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion according to claim 1, characterized in that, In STEP7, the multi-scale feature interaction module consists of a top-down convolutional network and a bottom-up upsampling network.

4. The 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion according to claim 3, characterized in that, The structure of the top-down convolutional network is as follows: The three different scales of input feature maps from the 2D backbone network are respectively of , of , of ,in, For the number of channels, and Indicates the height and width of the feature; These feature maps are convolved, and a fusion operation of these feature maps at different scales is introduced. Each convolution operation is followed by a batch normalization layer and an activation function. Finally, a feature map that integrates all features is obtained. Specifically: ； ； ； ； ； in, It is the feature map after the input feature map has undergone convolutional preprocessing. for The feature map obtained after the first fusion process, for The feature map obtained after the first fusion process, for The feature map obtained after the second fusion process, For activation function, For batch normalization operations, This is a convolution operation.

5. The 4D millimeter-wave radar 3D target detection method based on multi-scale feature fusion according to claim 4, characterized in that, The structure of the bottom-up upsampling network is as follows: Feature map that integrates all features Upsampled to size Feature map Subsequently, in the feature map Feature maps are introduced sequentially during the upsampling operation. and feature map that integrates all features The fusion yields a size of Feature map Simultaneously, three feature maps from the 2D backbone network were upsampled to the same level as the feature maps. The feature maps are of the same size; finally, the above feature maps are concatenated along the channel dimension to obtain the final output feature. Specifically: ； ； ； ； ； ； ； in, for The feature map obtained after the first upsampling process, for The feature map obtained after the first fusion upsampling process, for The feature map obtained after the first fusion upsampling process, for The feature map obtained after the upsampling operation for The feature map obtained after the upsampling operation for The feature map obtained after the upsampling operation For upsampling operation, This is for splicing operations.

Images