A view feature conversion method, system, storage medium and product

Abstract

The present invention relates to the field of 3D object detection, and in particular to a view feature conversion method, system, storage medium and product, S1: constructing a data set; S2: extracting features from the data set to obtain a feature image; S3: performing two convolution operations on the feature image to obtain a first feature and a second feature; S4: obtaining a depth prediction through the first feature, and obtaining a height prediction and a width prediction through the second feature; S5: extracting a feature X from the feature image using a compressed feature extraction module; S6: using the depth prediction, the height prediction, the width prediction and the feature X as inputs of a feature-guided polar coordinate position encoding module to calculate and obtain a position encoding; S7: creating a learnable BEV embedding; S8: performing a cross-attention operation on feature X, feature X, the position encoding and the learnable BEV embedding to output preliminary features; S9: further extracting features from the preliminary features through a depth-separable convolution block to obtain final features; and reducing the complexity of calculation.

Description

Technical Field

[0001] The present invention relates to the field of 3D object detection, and in particular to a view feature conversion method, system, storage medium and product. Background Art

[0002] At present, the camera-bird's-eye view view feature conversion method has high computational complexity and requires special engineering operations for deployment. It is not friendly to devices with limited computing resources such as edge devices, which affects its application deployment in actual scenarios. The view feature conversion based on the LSS method utilizes the multi-threading mechanism of CUDA and relies on custom operators that can only be used on specific devices. The Transformer-based method introduces quadratic complexity relative to the input resolution and uses multi-layer decoder operations to calculate the bird's-eye view representation, which seriously affects the processing speed. Summary of the invention

[0003] The object of the present invention is to provide a view feature conversion method, system, storage medium and product with reduced complexity.

[0004] In order to achieve the above object, the present invention adopts such technical solution:

[0005] A method for converting view features, specifically comprising the following steps:

[0006] S1: Multiple cameras with different perspectives carried by the autonomous driving vehicle collect image data of the same location respectively, and the collected image data are used to construct a data set DataSet;

[0007] S2: Input the dataset DataSet into the backbone network for initial feature extraction, obtain the feature map, input the feature map into the feature pyramid for feature extraction, and obtain the feature image img;

[0008] S3: Input the feature image img into two different convolution modules for convolution operation to obtain the first feature and the second feature;

[0009] S4: Apply the Softmax function to the first feature along the channel (C) dimension to obtain the depth prediction value d_pre, and apply the Softmax function to the second feature along the height (H) and width (W) dimensions to obtain the height prediction value h_pre and the width prediction value w_pre;

[0010] S5: Extract feature X from the feature image img using the compression feature extraction module;

[0011] The specific operation of the compression feature extraction module is as follows: Taking the feature image of step S2 As the input of the compression feature extraction module, the maximum pooling and average pooling operations are first used to compress the feature image img in the vertical and horizontal directions respectively to obtain the compressed feature and ; The compressed features and The height prediction feature and width prediction feature of the query and the corresponding feature image img are used as the key and value of two cross attentions respectively, and two independent cross attention calculations are performed to obtain two features after cross attention calculation; the features after cross attention calculation are spliced ​​along the feature dimension to form a fused feature ; Finally, the self-attention operation is used to fused the features Further information enhancement is performed to improve the performance of the model. The entire feature extraction process can be expressed by the following formula:

[0012] ;

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] Among them, Conv, MaxPool, AvgPool, Cross_Attention, and Self_Attention represent convolution operation, maximum pooling operation, average pooling operation, cross attention operation, and self-attention operation respectively. is the H-dimensional vector of features, is the W-dimensional vector of the feature. Dim=2 in the Concat operation means concatenation in the H and W dimensions.

[0020] S6: The depth prediction value d_pre, the height prediction value h_pre, the width prediction value w_pre and the fused feature X are used as the input of the feature-guided polar coordinate position encoding module, and the position encoding calculation is performed to obtain the position encoding fppe_pos;

[0021] The specific operation of the feature-guided polar coordinate position encoding module is as follows: first, the coordinates of the fused feature X are uniformly discretized to generate a cone coordinate point P; the cone coordinate point P is converted into a lidar coordinate system using camera internal and external parameters to form a 3D coordinate point Coor; within a predefined range, the 3D coordinate point Coor is converted into a voxel coordinate system; with the center coordinates of the autonomous driving vehicle as the origin, the radial distance d and the sine and cosine values ​​of the voxel coordinates in the polar coordinate system are calculated according to the voxel coordinates corresponding to the image features. and ; Radial distance d and sine and cosine values and Generate horizontal position code by encoding with sine and cosine position encoder Vertical Position Encoding ; The depth prediction value d_pre, height prediction value h_pre, and width prediction value w_pre are used as weighted values ​​of radial distance and sine and cosine values ​​in polar coordinate coding; the horizontal position coding and vertical position coding are concatenated and aggregated using a multi-layer perceptron to obtain the final polar coordinate position coding ; The feature-guided polar coordinate position encoding fppe_pos can be expressed as: att×C The fused feature X is processed through 1×1 convolution, batch normalization, nonlinear activation, spatial dimension processing, feature splitting and attention weight calculation to finally generate the attention vector , the attention vector Multiply it with the polar position code to generate the feature-guided polar position code, and add it to the fused feature X to obtain the key value (k) of the Transformer decoder. The fused feature X is used as the value (v) of the Transformer decoder. The entire feature-guided polar position code calculation process can be described by the following formula:

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] Among them, K and T are the rotation matrix and translation vector that convert the coordinates of the nth view into the unified lidar coordinate system, I is the camera internal parameter, P is the view cone coordinate point, x and y are the x and y coordinates in the voxel coordinate system, respectively. represents sine-cosine position encoding, MLP represents multi-layer perceptron, Concat(dim=-1) represents concatenation along the channel dimension, Concat(dim=2) represents concatenation along the image dimension, BatchNorm represents normalization operation, Conv 1×1 represents a 1×1 convolution operation, ReLU represents the activation function, split(dim=2) represents splitting along the image size, D =60 is the predefined depth range, express Activation function;

[0038] S7: Use the Embedding operation to create a learnable BEV embedding;

[0039] S8: Encode the fused feature X, the fused feature X and the polar coordinate position The sum of and the learnable BEV embedding are used as the value (v), key (k) and query (q) of the cross attention calculation respectively, and the cross attention operation is performed to output the preliminary BEV features;

[0040] S9: The preliminary BEV features are further extracted through a depth-wise separable convolution block to obtain the final BEV features.

[0041] Preferably, step S1 uses six cameras with different viewing angles, namely, left front, front, right front, left rear, rear and right rear, to capture images; camera parameters parameter and true value label Label.

[0042] Preferably, the first detection head is used to calculate the monocular 3D detection prediction value of the feature image img, and the loss is calculated between the prediction value and the true value label to obtain the loss value ;

[0043] Separate the width feature from feature X, use the second detection head to calculate the monocular 3D detection prediction value of the width feature, and calculate the loss value by comparing the prediction value with the true value label. ;

[0044] The final BEV features are further extracted through Bev_Enconder encoding, and the multi-view Figure 3 D detects the predicted value and calculates the loss between the predicted value and the true value label .

[0045] Preferably, the first detection head is an Fcos3d detection head, the second detection head is a Width_Fcos3d detection head, and the third detection head is a CenterPoint detection head.

[0046] Preferably, the total loss calculation formula is as follows:

[0047] ;

[0048] in, For multi-view Figure 3 D is the loss of object detection, and represents the loss of monocular 3D target detection, Used for gradient calculation to optimize the model.

[0049] Preferably, step S3 uses 1×1 convolution and 3×3 convolution to perform convolution operations.

[0050] Preferably, in step S5, a lightweight convolutional network is used to predict the depth prediction value d_pre, the height prediction value h_pre and the width prediction value w_pre of the image feature.

[0051] A view feature conversion system includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, any one of the above methods is implemented.

[0052] A storage medium comprises a memory, wherein a computer program is stored in the memory, and when the computer program is executed by a processor, any of the above methods is implemented.

[0053] A view feature conversion product includes a computer program, and when the computer program is executed by a processor, it implements any of the above methods.

[0054] By adopting the aforementioned design scheme, the beneficial effects of the present invention are as follows: the present invention designs a compressed feature extraction module and feature-guided polar coordinate position coding, which reduces the complexity of calculation and supplements the missing information while compressing the feature information; the feature-guided polar coordinate position coding well represents the position structure of the feature in space, making it more intuitive when representing the spatial structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the algorithm of the present invention in the 3D object detection framework;

[0056] Figure 2 is a flow chart of the view feature conversion algorithm of the present invention;

[0057] Figure 3 Schematic diagram of the structure of the depth-wise separable convolution module of the present invention;

[0058] Figure 4 It is an operation flow chart of the compression feature extraction module of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] A view feature conversion method, such as Figure 1-Figure 2 As shown, the specific steps include the following:

[0061] S1: Multiple cameras with different perspectives carried by the autonomous driving vehicle collect image data of the same location respectively, and the collected image data are used to construct a data set DataSet;

[0062] In this embodiment, by acquiring the location information of the autonomous driving vehicle, the acquired image data is associated with the location information, and the image data collected at the same location is constructed into a data set DataSet.

[0063] In this embodiment, six cameras with different viewing angles, namely, left front, front, right front, left rear, rear and right rear, are used to capture images; camera parameters parameter and true value label Label.

[0064] S2: Input the data set DataSet into the backbone network for initial feature extraction, obtain a feature map, input the feature map into the feature pyramid (FPN) for feature extraction, and obtain a feature image img; the backbone network used in this embodiment is ResNet.

[0065] S3: Input the feature image img into two different convolution modules for convolution operation to obtain the first feature and the second feature;

[0066] In this implementation, step S3 uses 1×1 convolution and 3×3 convolution to perform convolution operations;

[0067] S4: Apply the Softmax function to the first feature along the channel (C) dimension to obtain the depth prediction value d_pre, and apply the Softmax function to the second feature along the height (H) and width (W) dimensions to obtain the height prediction value h_pre and the width prediction value w_pre;

[0068] S5: Using the compressed feature extraction module (CFE), extract feature X from the feature image img; Figure 4 shown.

[0069] In this embodiment, the learnable BEV embedding is used as the query vector, combined with the Transformer decoding layer to calculate the bird's-eye view features.

[0070] Assume that the multi-view feature resolution is ( ), the BEV query resolution is ( ), whose computational complexity is ,This high computational complexity seriously restricts the computational efficiency of the model. In order to solve the above problems, the present invention proposes a compression feature extraction module. The module aims to retain the main features of the image while filtering out redundant information and significantly reduce the computational complexity when calculating BEV features without sacrificing performance.

[0071] The specific operation of the compression feature extraction module is as follows: Taking the feature image of step S2 As the input of the compression feature extraction module. First, use the maximum pooling (MaxPool) and average pooling (AvgPool) operations to compress the feature image img in the vertical and horizontal directions respectively, and obtain the compressed feature and The compressed features and The height prediction feature and width prediction feature of the query and the corresponding feature image img are used as the key and value of two cross attentions respectively, and two independent cross attention calculations are performed to obtain two features calculated after cross attention. This step is to supplement the information that may be lost during the feature compression process. The features calculated after cross attention are spliced ​​along the feature dimension to form a fused feature Finally, the self-attention mechanism is used to fused the features Further information enhancement is performed to improve the performance of the model. At this point, the computational complexity is reduced to The entire feature extraction process can be expressed by the following formula:

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] Among them, Conv, MaxPool, AvgPool, Cross_Attention, and Self_Attention represent convolution operation, maximum pooling operation, average pooling operation, cross attention operation, and self-attention operation respectively. is the H-dimensional vector of features, is the W-dimensional vector of features. dim=2 in the Concat operation means concatenation in the H and W dimensions.

[0080] S6: The depth prediction value d_pre, the height prediction value h_pre, the width prediction value w_pre and the fused feature X are used as the input of the feature-guided polar coordinate position encoding module (FPPE), and the position encoding is calculated to obtain the position encoding fppe_pos;

[0081] In the Transformer decoding layer, position encoding is the core of understanding the relationship between sequence elements. In order to enhance the model's ability to capture such positional relationships, this paper proposes a feature-guided polar coordinate position encoding module, which incorporates image features into the generation process of 3D position embedding by introducing implicit visual priors.

[0082] First, the coordinates of the fused feature X are uniformly discretized to generate the view cone coordinate point P; the view cone coordinate point P is transformed into the lidar coordinate system using the camera internal and external parameters (K, I, T) to form a 3D coordinate point Coor; within a predefined range, the 3D coordinate point Coor is converted to the voxel coordinate system. Taking the center coordinates of the autonomous driving vehicle as the origin, the radial distance d and the sine and cosine values ​​cosθ and sinθ of the voxel coordinates in the polar coordinate system are calculated according to the voxel coordinates corresponding to the image features. The radial distance d and the sine and cosine values ​​cosθ and sinθ are encoded by the sine and cosine position encoder to generate the horizontal position encoding Vertical Position Encoding The present invention uses a lightweight convolutional network to predict the depth prediction value d_pre, height prediction value h_pre and width prediction value w_pre of the image feature, and these prediction values ​​are used as weighted values ​​of radial distance and sine and cosine values ​​in polar coordinate coding. The horizontal position coding and vertical position coding are spliced ​​and aggregated using a multi-layer perceptron to obtain the final polar coordinate position coding. The feature-guided polar coordinate position encoding fppe_pos can be expressed as: att×C The fused feature X is processed through 1×1 convolution, batch normalization, nonlinear activation, spatial dimension processing, feature splitting and attention weight calculation to finally generate the attention vector , the attention vector Multiply it with the polar position code to generate the feature-guided polar position code, and add it to the fused feature X to obtain the key value (k) of the Transformer decoder. The fused feature X is used as the value (v) of the Transformer decoder. The entire feature-guided polar position code calculation process can be described by the following formula:

[0083] ;

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] Among them, K and T are the rotation matrix and translation vector that convert the coordinates of the nth view into the unified lidar coordinate system, I is the camera internal parameter, P is the view cone coordinate point, x and y are the x and y coordinates in the voxel coordinate system, respectively. represents sine-cosine position encoding, MLP represents multi-layer perceptron, Concat(dim=-1) represents concatenation along the channel dimension, Concat(dim=2) represents concatenation along the image dimension, BatchNorm represents normalization operation, Conv 1×1 represents a 1×1 convolution operation, ReLU represents the activation function, split(dim=2) represents splitting along the image size, D =60 is the predefined depth range, express Activation function.

[0099] S7: Use the Embedding operation to create a learnable BEV embedding;

[0100] S8: Encode the fused feature X, the fused feature X and the polar coordinate position The sum of and the learnable BEV embedding are used as the value (v), key (k) and query (q) of the cross attention calculation respectively, and the cross attention operation is performed to output the preliminary BEV features;

[0101] S9: The preliminary BEV characteristics are obtained by Figure 3 The depthwise separable convolutional blocks shown further extract features to obtain the final BEV features.

[0102] As a preferred method of this embodiment, the first detection head is used to calculate the monocular 3D detection prediction value of the feature image img, and the loss is calculated between the prediction value and the true value label to obtain the loss value ;

[0103] Separate the width feature from feature X, use the second detection head to calculate the monocular 3D detection prediction value of the width feature, and calculate the loss value by comparing the prediction value with the true value label. ;

[0104] The BEV features are further extracted through Bev_Enconder (ResNet+FPN) encoding, and the multi-view detection head is used to obtain Figure 3 D detects the predicted value and calculates the loss between the predicted value and the true value label .

[0105] In this implementation, the first detection head is an Fcos3d detection head, the second detection head is a Width_Fcos3d detection head, and the third detection head is a CenterPoint detection head.

[0106] The total loss calculation formula of the present invention is as follows:

[0107] ;

[0108] in, For multi-view Figure 3 D is the loss of object detection, and represents the loss of monocular 3D target detection, Used for gradient calculation to optimize the model.

[0109] This embodiment also provides a system for implementing the above method.

[0110] A view feature conversion system includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, any of the above methods is implemented.

[0111] This embodiment also provides a storage medium storing a computer program that can implement the above method.

[0112] A storage medium comprises a memory, wherein a computer program is stored in the memory, and when the computer program is executed by a processor, any of the above methods is implemented.

[0113] This embodiment also provides a computer product that can implement the above method.

[0114] A view feature conversion product includes a computer program, and when the computer program is executed by a processor, it implements any of the above methods.

[0115] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A view feature conversion method, characterized in that: The specific steps include: S1: Multiple cameras with different perspectives carried by the autonomous driving vehicle collect image data of the same location respectively, and the collected image data are used to construct a data set DataSet; S2: Input the dataset DataSet into the backbone network for initial feature extraction, obtain the feature map, input the feature map into the feature pyramid for feature extraction, and obtain the feature image img; S3: Input the feature image img into two different convolution modules for convolution operation to obtain the first feature and the second feature; S4: Apply the Softmax function to the first feature along the channel C dimension to obtain the depth prediction value d_pre, and apply the Softmax function to the second feature along the height H and width W dimensions to obtain the height prediction value h_pre and the width prediction value w_pre; S5: extract features from the feature image img using the compression feature extraction module; The specific operation of the compression feature extraction module is as follows: Taking the feature image of step S2 As the input of the compression feature extraction module, the maximum pooling and average pooling operations are first used to compress the feature image img in the vertical and horizontal directions respectively to obtain the compressed feature and The compressed feature img W with img H As two cross-attention queries, the H-dimensional vector img(H) and the W-dimensional vector img(W) of the features extracted from the corresponding feature image img are used as the keys and values ​​of the two cross-attentions respectively, and two independent cross-attention calculations are performed to obtain two features after cross-attention calculations; the features after cross-attention calculations are spliced ​​along the feature dimension to form a fused feature Finally, the self-attention operation is used to further enhance the information of the fused feature X to obtain the feature X′. The entire feature extraction process is expressed by the following formula: img′=Conv 1×1 (img); img W =MaxPool(img′); img H =AvgPool(img′); img′ W =Cross_Attention(img W ,img(H),img(H)); img′ H =Cross_Attention(img H ,img(W),img(W)); X=Concat((img′ H ,img′ W ),dim=2); X′=Self_Attention(X,X,X); Among them, Conv, MaxPool, AvgPool, Cross_Attention, and Self_Attention represent convolution operation, maximum pooling operation, average pooling operation, cross attention operation, and self-attention operation respectively, img(H) is the H-dimensional vector of the extracted features, img(W) is the W-dimensional vector of the extracted features, and dim=2 in the Concat operation means concatenation in the H and W dimensions; S6: Take the depth prediction value d_pre, the height prediction value h_pre, the width prediction value w_pre and the feature X′ as inputs of the feature-guided polar coordinate position encoding module, and perform position encoding calculation to obtain the position encoding fppe_pos; The specific operation of the feature-guided polar coordinate position encoding module is as follows: first, the coordinates of the feature X′ are uniformly discretized to generate a view cone coordinate point P; the view cone coordinate point P is converted into a lidar coordinate system using camera internal and external parameters to form a 3D coordinate point Coor; within a predefined range bound, the 3D coordinate point Coor is converted into a voxel coordinate system; taking the center coordinates of the autonomous driving vehicle as the origin, according to the voxel coordinates corresponding to the image features, the radial distance d and the sine and cosine values ​​cosθ and sinθ of the voxel coordinates in the polar coordinate system are calculated; the radial distance d and the sine and cosine values ​​cosθ and sinθ are encoded by a sine and cosine position encoder to generate a horizontal position code With vertical position encoding The depth prediction value d_pre, height prediction value h_pre, and width prediction value w_pre are used as weighted values ​​of radial distance and sine and cosine values ​​in polar coordinate coding; the horizontal position coding and vertical position coding are concatenated and aggregated using a multi-layer perceptron to obtain the final polar coordinate position coding. The feature-guided polar coordinate position encoding fppe_pos is expressed as: att×C; the feature X′ is finally generated through 1×1 convolution, batch normalization, nonlinear activation, spatial dimension processing, feature splitting and attention weight calculation. Attention Vector Multiply it with the polar position code to generate the feature-guided polar position code, and add it to the feature X′ to obtain the key value k of the Transformer decoder. The feature X′ is used as the value v of the Transformer decoder. The entire feature-guided polar position code calculation process is described by the following formula: Coor i,j,m =K n ·I -1 ·P i,j,m +T n ; bound:[min,max,intveral]; Coor′ i,j,m =(Coor i,j,m -min) / interval; d′=ψ(d),θ=ψ(Concat(cosθ,sinθ)); C H =Concat([d h ,θ h ],dim−1),C W =Concat([d w ,θ w ],dim⼝-1); C=MLP(Concat([C H ,C W ],dim=2)); temp=ReLU(BatchNorm(Conv 1×1 (X′))); temp h ,temp w =split(temp,[h,w],dim=2); temp h ′=Sigmoid(Conv 1×1 (temp h )), temp′ w =Sigmoid(Conv 1×1 (temp w )); to=Concat([temp′ h ,temp′ w ],dim=2); fppe_pos=att×C; k = X′ + fppe_pos; Among them, K and T are the rotation matrix and translation vector that convert the coordinates of the nth view into the unified lidar coordinate system, I is the camera internal parameter, P is the cone coordinate point, x and y are the x and y coordinates in the voxel coordinate system, ψ represents the encoding using the sine-cosine position encoder, MLP represents the multi-layer perceptron, Concat([d w ,θ w ], dim=-1) means concatenation along the channel dimension, Concat([C H ,C W ], dim=2) indicates splicing along the image size, BatchNorm indicates normalization operation, Conv 1×1 represents a 1×1 convolution operation, ReLU represents the activation function, split(temp,[h,w],dim=2) represents splitting along the image size, D=60 is the predefined depth range, and Sigmoid() represents the Sigmoid activation function; S7: Use the Embedding operation to create a learnable BEV embedding; S8: Take the feature X′, the sum of the feature X′ and the polar coordinate position encoding fppe_pos, and the learnable BEV embedding as the value, key value, and query for the cross attention calculation, perform the cross attention operation, and output the preliminary BEV features; S9: The preliminary BEV features are further extracted through a depth-wise separable convolution block to obtain the final BEV features.

2. The view feature conversion method according to claim 1, wherein: Step S1 uses six cameras with different viewing angles, namely, left front, front, right front, left rear, rear and right rear, to collect images; and defines camera parameters parameter and true value labels Label.

3. The view feature conversion method according to claim 2, wherein: The first detection head is used to calculate the monocular 3D detection prediction value of the feature image img, and the loss is calculated by comparing the monocular 3D detection prediction value with the true value label Label to obtain the loss value Loss aux1 ; Separate the width feature from the feature X′, use the second detection head to calculate the monocular 3D detection prediction value of the width feature, and calculate the loss value Loss by comparing the monocular 3D detection prediction value of the width feature with the true value label Label aux2 ; The final BEV feature is further extracted through Bev_Enconder encoding, and the multi-view 3D detection prediction value is obtained through the third detection head. The multi-view 3D detection prediction value and the true value label Label are used to calculate the loss Loss bev .

4. The view feature conversion method according to claim 3, wherein: The first detection head is an Fcos3d detection head, the second detection head is a Width_Fcos3d detection head, and the third detection head is a CenterPoint detection head.

5. The view feature conversion method according to claim 3, characterized in that: The total loss is calculated as follows: Loss all =Loss bev +Loss aux1 +Loss aux2 ; Among them, Loss bev Loss is the loss of multi-view 3D object detection. aux1 and Loss aux2 Represents the loss of monocular 3D target detection, Loss all Used for gradient calculation to optimize the model.

6. The view feature conversion method according to claim 1, wherein: Step S3 uses 1×1 convolution and 3×3 convolution to perform convolution operations.

7. The view feature conversion method according to claim 1, wherein: In step S5, a lightweight convolutional network is used to predict the depth prediction value d_pre, the height prediction value h_pre, and the width prediction value w_pre of the image feature.

8. A view feature conversion system, comprising a memory and a processor, wherein a computer program is stored in the memory, characterized in that: When the computer program is executed by the processor, the view feature conversion method described in any one of claims 1 to 7 is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the view feature conversion method described in any one of claims 1 to 7 is implemented.

10. A program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the view feature conversion method described in any one of claims 1 to 7 is implemented.

Images