A monocular three-dimensional target detection method based on convolution attention and feature decoupling

Abstract

The application relates to a monocular three-dimensional target detection method based on convolution attention and feature decoupling, which comprises the following steps: given an input image, outputting a feature map through a backbone network, and implicitly learning depth features under the auxiliary supervision of a depth map by using a depth estimator; inputting visual features and depth features into a visual multi-scale detail convolutional encoder and a depth multi-scale detail convolutional encoder respectively for processing, the multi-scale detail convolutional encoder integrating a multi-scale asymmetric convolution attention module to aggregate local features of the input image and encode the aggregated local features; using a double-encoder-double-decoder structure of a decoupled feature guided self-attention model to respectively encode and decode spatial and appearance information of the input image; fusing outputs of a depth branch and a visual branch; adopting a predefined two-dimensional-three-dimensional anchor single-stage detector to regress a bounding box; predicting a bounding box of each anchor point; and restoring the bounding box.

Description

Technical Field

[0001] This invention relates to a monocular three-dimensional target detection method. Background Technology

[0002] Monocular 3D object detection is a challenging task in the field of autonomous driving. Initially, some monocular methods used only one image as input [1] and made impressive progress by taking advantage of the geometric constraints between two and three dimensions. Although monocular 3D object detection has the advantage of low cost, it is difficult to estimate depth from a single image, resulting in the performance of monocular 3D object detection still being far from satisfactory.

[0003] In the field of monocular 3D object detection, methods that rely solely on images initially depend on a single image to predict the target. However, the lack of depth information in the image poses a challenge. Some methods rely on geometric consistency for prediction to address this limitation. Reference [2] combines the geometric relationships of 2D and 3D projections to construct a 3D target region proposal network. Reference [3] further improves 3D detection performance based on the exploration of pairwise spatial relationships. Reference [4] first introduces the bottom surface of the object's 3D bounding box as a ground plane to mitigate the interference of object-irrelevant attributes. Reference [5] treats monocular target depth estimation as an asymptotic refinement problem and proposes a joint semantic and geometric cost volume to model the depth error. To address the limitations of monocular 3D object detection due to the lack of depth cues, researchers have proposed additional methods that utilize depth information during training [6]. Reference [7] focuses on image fusion and estimates depth using a specially designed convolutional network. Reference [8] utilizes a graphical model to efficiently extract contextual information from adjacent point clouds. Reference [9] reconstructs the 2D coordinates of the image by projecting 3D coordinates and learns the target geometric information in a self-supervised manner.

[0004] Inspired by the powerful self-attention mechanism and global feature extraction capability of Visual Transformers (ViTs)

[10] , some works have successfully applied the Visual Self-Attention Model to the monocular target detection task in autonomous driving scenarios, thereby further improving the detection accuracy. Reference

[11] proposed the first monocular 3D target detection network based on the self-attention model, which effectively integrates visual and depth features and improves the accuracy of monocular 3D target detection. In addition, Reference

[12] proposed to detect objects through the encoder-decoder paradigm and use the Hungarian matching algorithm for output prediction. Based on the visual encoder and decoder of

[12] , Reference

[13] proposed a depth encoder and a depth guided decoder for adaptive scene-level depth understanding, which significantly improved the accuracy of monocular 3D target detection.

[0005] While monocular 3D object detection methods based on visual self-attention models have achieved some success, there is a current trend towards further improvements. One possible solution is to introduce convolutional local features into visual self-attention, giving it a degree of offset invariance, scale invariance, and distortion invariance. Furthermore, all existing visual self-attention-based models input visual and depth features into the same decoder for processing. Due to the nature of monocular tasks, inaccurate depth information can interfere with the model's learning of other information.

[0006] References:

[0007] [1]Ku J,Pon AD,Waslander S L.Monocular 3d object detectionleveraging accurate proposals and shape reconstruction[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition.2019:11867-11876.

[0008] [2]Brazil G, Liu

[0009] [3]Chen Y,Tai L,Sun K,et al.Monopair:Monocular 3d object detection using pairwise spatial relationships[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2020:12093-12102.

[0010] [4]Qin Z,Li X.Monoground:Detecting monocular 3d objects from theground[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition.2022:3793-3802.

[0011] [5]Lian Q,Li P,Chen X.Monojsg:Joint semantic and geometric costvolume for monocular 3d object detection[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2022:1070-1079.

[0012] [6]WangY,Chao W L,Garg D,et al.Pseudo-lidar from visual depthestimation:Bridging the gap in 3d object detection for autonomous driving[C] / / Proceedings ofthe IEEE / CVF Conference on Computer Vision and PatternRecognition.2019:8445-8453.

[0013] [7]Ding M,Huo Y,Yi H,et al.Learning depth-guided convolutions formonocular 3d object detection[C] / / Proceedings of the IEEE / CVF Conference oncomputer vision and pattern recognition workshops.2020:1000-1001.

[0014] [8]Wang L,Du L,Ye X,et al.Depth-conditioned dynamic messagepropagation for monocular 3d object detection[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2021:454-463.

[0015] [9]Chen H,Huang Y,Tian W,et al.Monorun:Monocular 3d object detectionby reconstruction and uncertainty propagation[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2021:10379-10388.

[0016]

[10] Han K,Wang Y,Chen H,et al.A survey on vision transformer[J].IEEEtransactions on pattern analysis and machine intelligence,2022,45(1):87-110.

[0017]

[11] Huang K C,Wu T H,Su H T,et al.Monodtr:Monocular 3d objectdetection with depth-aware transformer[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2022:4012-4021.

[0018]

[12] Carion N, Massa F, Synnaeve G, et al. End-to-end object detection with transformers[C] / / European conference on computer vision. Cham: Springer International Publishing, 2020: 213-229.

[0019]

[13] Zhang R, Qiu H, Wang T, et al. MonoDETR: depth-guided transformer for monocular 3D object detection [J]. arXiv preprint arXiv: 2203.13310, 2022.

[0020]

[14] Huang KC, Wu TH, Su HT, et al.Monodtr: Monocular 3d object detection with depth-aware transformer[C] / / Proceedings of the IEEE / CVFConference on ComputerVision and Pattern Recognition.2022:4012-4021. Summary of the Invention

[0021] This patent addresses the aforementioned problems of existing technologies by proposing a monocular 3D object detection method based on multi-scale asymmetric convolutional attention and feature decoupling. This method introduces asymmetric convolution into a self-attention model with decoupled depth and visual features. A multi-scale detail convolutional encoder aggregates local features of the input image and uses depthwise convolution with multiple convolutional kernels of different shapes to encode features, thereby obtaining more details in the image. The decoupled structure allows visual and depth features to learn independently without interfering with each other. Furthermore, the combination of convolution and self-attention models enables this method to possess both local and global properties simultaneously. Experimental results on the KITTI dataset demonstrate that this method exhibits superior performance compared to other state-of-the-art methods. The technical solution is as follows:

[0022] A monocular 3D object detection method based on convolutional attention and feature decoupling includes the following steps:

[0023] Step 1: Given an input image with a resolution of H×W, the backbone network outputs a feature map F. Using F as the original feature map, a series of convolutional layers are used to generate visual features f. V =Conv(F), and then implicitly learn deep features f using a depth estimator under the supervision of the depth map. D The method is as follows:

[0024] For the original feature F, two convolutional layers are used to predict the probability of a discrete depth interval D. This probability represents the confidence level that the depth value of each pixel belongs to a certain depth interval. Linearly increasing discretization is used to discretize the true depth value from the continuous space to the discrete space d. i , i is the depth range index, d i This represents the i-th discrete interval, counted from shallowest to deepest.

[0025] Let the intermediate feature map X = Conv(F) represent the initial depth-aware features. The feature center of each depth interval, i.e., the depth prototype, is calculated by aggregating the depth-aware features of each pixel belonging to the specified interval. Group convolution is used to generate the predicted depth intervals D. The number of intervals is reduced from N to N' = N / r using a set scale r. The depth prototype F d It is generated by weighting the features X' of all pixels according to their respective probabilities to the depth category m. Let X... i ' represents the feature of the i-th depth interval pixel in X' = Conv(X), where L is the pixel set in the feature map X', based on the prototype F. d The reconstructed new deep features are denoted as f. D ;

[0026] Step two involves inputting visual and depth features into a visual multi-scale detail convolutional encoder and a depth multi-scale detail convolutional encoder, respectively. The multi-scale detail convolutional encoder integrates a multi-scale asymmetric convolutional attention module to aggregate local features of the input image and encodes the aggregated local features, as follows:

[0027] The input visual and depth features are each processed through a multi-scale asymmetric convolutional attention module, resulting in the attention map and output sum f of the multi-scale asymmetric convolutional attention module. out ;

[0028] In the multi-scale asymmetric convolutional attention module, the input features are first aggregated by asymmetric convolution to aggregate local information; then multi-branch asymmetric convolution is used to capture multi-scale contextual information, and then 1×1 convolution is used to model the channel dimension.

[0029] The output f of the multi-scale asymmetric convolutional attention module outThe final output f of the multi-scale detail convolutional encoder is generated through a feedforward network and a batch normalization layer. c ;

[0030] Step 3: The spatial and visual information of the input image is encoded and decoded using the dual encoder-dual decoder structure of the decoupled feature-guided self-attention model. The decoupled feature-guided self-attention model has two branches: depth and vision. Both branches use the same structure, and their parameters are trained separately. The two branches of the decoupled feature-guided self-attention model are respectively fed by the output of the depth multi-scale detail convolutional encoder. and the output of the visual multi-scale detail convolutional encoder As input;

[0031] Step four: Use the cross-attention-guided fusion module to process the output F of the deep branch. D and the output F of the visual branch V Perform fusion; in the cross-attention-guided fusion module, F D As input query Q FD =Linear(F D ), F V As K FV V FV =Linear(F V The input is fed into a cross-attention layer; the output of the cross-attention layer is then passed through a layer normalization layer to obtain the final output F of the fusion module. fused :

[0032]

[0033] Step 5: A single-stage detector with predefined 2D-3D anchor points is used to regress the bounding box. Each predefined anchor point is represented by a 2D bounding box [x...]. 2d ,y 2d ,w 2d ,h 2d ] and 3D bounding box [x p ,y p ,z,w 3d ,h 3d ,l 3d The parameter composition of [x,θ], [x 2d ,y 2d ] and [x p ,y p [w] represents the center of the two-dimensional bounding box and the center of the three-dimensional object projected onto the image plane. 2d ,h 2d ] and [w 3d ,h 3d ,l 3d] represents the physical dimensions of the 2D and 3D bounding boxes, respectively; z represents the depth of the 3D object center; θ represents the observation angle. During training, all ground truth values ​​are projected onto the 2D space to calculate the intersection of all 2D anchor points. Anchor points with an intersection greater than 0.5 are selected, and the corresponding 3D boxes are optimized accordingly.

[0034] Step 6: Predict the 2D bounding box for each anchor point [t] x ,t y ,t w ,t h ] 2d and 3D bounding box [t x ,t y ,t w ,t h ,t l ,t z ,t θ ] 3d It parameterizes the residual values ​​of 2D and 3D bounding boxes, predicts classification scores, and outputs the recovered bounding boxes based on anchor points and network predictions.

[0035] Furthermore, in step one, linearly increasing discretization is used to discretize the true depth value from the continuous space to the discrete space d. i :

[0036]

[0037] Where N is the number of depth intervals, [d min d max ] represents the depth range; pixels with depth values ​​outside this range will be marked as invalid and will not be used for optimization during training. i is the depth range index, d... i This represents the i-th interval, counted from shallowest to deepest.

[0038] Furthermore, in step one, the number of depth intervals N is set to 96, and the depth range [d] min d max The value is set to [1, 80], and r = 2.

[0039] Furthermore, in step two, the asymmetric convolution is represented as:

[0040]

[0041] Where BN represents the batch normalization operation, γ i and β i These are learnable parameters in the BN operation, i = 1, 2, 3.

[0042] Furthermore, the multi-scale asymmetric convolutional attention module mentioned in step two is as follows:

[0043]

[0044] f out =Attention × f

[0045] Where f represents the input feature f D or f V Asy_Conv represents asymmetric convolution; Scale i Let i∈{0,1,2,3} represent the i-th branch; Attention and f out These represent the attention map and output of the multi-scale asymmetric convolutional attention module, respectively.

[0046] Furthermore, in step three, for the depth branch, the method is as follows:

[0047] 1) The output of the depth multi-scale detail convolutional encoder As input features, they are fed into deep branches of the self-attention model guided by decoupled features;

[0048] 2) In the encoder of the depth branch, Encoding is performed using a self-attention layer and a layer normalization layer:

[0049]

[0050]

[0051]

[0052] Where Linear represents linear transformation, softmax is an activation function, C represents the dimension of the input features, LN represents layer normalization operation, and A represents the attention score. This represents the encoder output in the depth branch;

[0053] 3) In the decoder of the deep branch, As input query Position code P as K P V P =Linear(P) is input into a cross-attention layer; the output of the cross-attention layer is decoded through a self-attention layer and a layer normalization layer to obtain the output F of the depth branch. D The implementation method of the cross-attention layer is as follows:

[0054]

[0055] Among them, A D This represents the attention score of the cross-attention layer;

[0056] For the visual branch, process it in the same way to obtain the output F of the visual branch. V .

[0057] Furthermore, in step six, the output bounding box is restored as follows:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063] Where (^) represents the restoration parameter of the three-dimensional object; for the two-dimensional boundary center [x 2d ,y 2d ] and 3D projection center [x p ,y p The same anchoring center was applied. Attached Figure Description

[0064] Figure 1 Overall structure diagram of the proposed method

[0065] Figure 2 Visualize the 3D bounding boxes of car categories on the KITTI dataset. Detailed Implementation

[0066] The method of this invention belongs to supervised learning. First, the model needs to be trained under supervision to obtain the optimal model, and then new data is used for detection. The overall structure of this invention is as follows: Figure 1 As shown. To make the technical solution of the present invention clearer, the specific embodiments of the present invention are further described below. The present invention is implemented in specific steps.

[0067] Step 1: Given an input image with a resolution of H×W, output a feature map F through the backbone network DLA-102. Using F as the original feature map, generate visual features f using a series of convolutional layers. V =Conv(F). Then, a depth estimator is used to implicitly learn deep features f under the supervision of the depth map. D

[14] The method is as follows:

[0068] Deep features f DThe specific generation process is as follows: For the original feature F, two convolutional layers are used to predict the probability of discrete depth intervals D, where N is the number of depth intervals, and this probability represents the confidence level of each pixel's depth value belonging to a certain depth interval. Linear incremental discretization is used to discretize the true depth value from continuous space to discrete space:

[0069]

[0070] Where i is the depth range index, d i This represents the i-th interval, counted from shallowest to deepest. The number of depth intervals N is set to 96, and the depth range is [d]. min d max The depth value is set to [1, 80]. Pixels with depth values ​​outside this range are marked as invalid and not used for optimization during training. The intermediate feature map X = Conv(F) represents the initial depth-aware features. To further enhance the depth representation, a center representation of the corresponding depth interval is introduced to enhance the features of each pixel. The feature center of each depth interval (i.e., the depth prototype) can be computed by aggregating the depth-aware features of each pixel belonging to the specified interval. In practice, group convolutions are first applied to generate the predicted depth intervals D, and the number of intervals is reduced from N to N' = N / r using a scale r. In this example, r = 2 is used to share similar depth cues and reduce computation. Depth prototype F d The representation can be generated by weighting the features X' of all pixels according to their probabilities to the depth class m:

[0071]

[0072] Where X' i Let L represent the feature of the i-th depth interval pixel in X' = Conv(X), where L is the pixel set in the feature map X'. It is the normalized probability of the m-th depth prototype. In this way, F... d It can represent the global context information for each depth range.

[0073] Furthermore, new deep features f are reconstructed based on deep prototype representations. D This allows each pixel to understand the representation of the depth range from the global view. The method for reconstructing depth features is as follows:

[0074] Step two involves inputting the visual and depth features into the visual multi-scale detail convolutional encoder and the depth multi-scale detail convolutional encoder, respectively. The multi-scale detail convolutional encoder integrates a multi-scale asymmetric convolutional attention module to aggregate local features of the input image and encode these features, as follows:

[0075] The input features are processed through a multi-scale asymmetric convolutional attention module:

[0076]

[0077] f out =Attention × f

[0078] Where f represents the input feature f D or f V Asy_Conv represents asymmetric convolution. Scale i Let i ∈ {0, 1, 2, 3} represent the i-th branch. Attention and f out These represent the attention map and output of the multi-scale asymmetric convolutional attention module, respectively.

[0079] In the multi-scale asymmetric convolutional attention module, input features are first aggregated to gather local information using a 5×5 asymmetric convolution; then, multi-branch asymmetric convolutions are used to capture multi-scale contextual information, where the kernel sizes of each branch are set to 7, 11, and 21, respectively; further, channel-dimensional correlation modeling is performed using 1×1 convolutions. The asymmetric convolution is represented as:

[0080]

[0081] Where BN represents batch normalization operation, and γ and β are learnable parameters in the BN operation;

[0082] The output f of the multi-scale asymmetric convolutional attention module out The final output f of the multi-scale detail convolutional encoder is generated through a feedforward network and a batch normalization layer. c =BN(FFN(f out )). Where FFN represents a feedforward network;

[0083] Step 3: The spatial and visual information of the input image is encoded and decoded separately using the dual encoder-dual decoder structure of the decoupled feature-guided self-attention model. The decoupled feature-guided self-attention model has two branches: depth and vision. Both branches use the same structure, and their parameters are trained separately. The two branches of the decoupled feature-guided self-attention model are respectively fed into the output of the depth multi-scale detail convolutional encoder. and the output of the visual multi-scale detail convolutional encoder As input. Taking deep branching as an example, the method is as follows:

[0084] 1) The output of the depth multi-scale detail convolutional encoder As input features, they are fed into deep branches of the self-attention model guided by decoupled features;

[0085] 2) In the encoder of the depth branch, Encoding is performed using a self-attention layer and a layer normalization layer:

[0086]

[0087]

[0088]

[0089] Where Linear represents a linear transformation, softmax is an activation function, C represents the dimension of the input features, LN represents the layer normalization operation, and A represents the attention score. This represents the encoder output in the depth branch;

[0090] 4) In the decoder of the deep branch, As input query Position code P as K P V P =Linear(P) is input into a cross-attention layer. Further, the output of the cross-attention layer is decoded by a self-attention layer and a layer normalization layer. The implementation method of the cross-attention layer is as follows:

[0091]

[0092] Among them, A D This represents the attention score of the cross-attention layer;

[0093] Step four: Use the cross-attention-guided fusion module to process the output F of the deep branch. D and the output F of the visual branch V Perform fusion. In the cross-attention-guided fusion module, F D As input query Q FD =Linear(F D ), F V As K FV V FV =Linear(F V The input is fed into a cross-attention layer. Further, the output of the cross-attention layer is passed through a layer normalization layer to obtain the final output F of the fusion module. fused :

[0094]

[0095] Step 5: A single-stage detector using predefined 2D-3D anchor points is used to regress the bounding box. Each predefined anchor point is represented by a 2D bounding box [x...].2d ,y 2d ,w 2d ,h 2d ] and 3D bounding box [x p ,y p ,z,w 3d ,h 3d ,l 3d The parameters of [x, θ] are composed of [x]. 2d ,y 2d ] and [x p ,y p [] represents the center of the two-dimensional bounding box and the center of the three-dimensional object projected onto the image plane. 2d ,h 2d ] and [w 3d ,h 3d ,l 3d ] represents the physical dimensions of the 2D and 3D bounding boxes, respectively. z represents the depth of the 3D object center. θ represents the observation angle. During training, we project all ground truth values ​​onto 2D space to calculate the intersection of all 2D anchor points. Anchor points with an intersection greater than 0.5 are selected, and their corresponding 3D bounding boxes are optimized.

[0096] Step 6, predict [t] for each anchor point x ,t y ,t w ,t h ] 2d and [t] x ,t y ,t w ,t h ,t l ,t z ,t θ ] 3d The residual values ​​of the 2D and 3D bounding boxes are parameterized, and classification scores are predicted. Based on the anchor points and network predictions, the output bounding boxes can be recovered as follows:

[0097]

[0098]

[0099]

[0100]

[0101]

[0102] Where (^) represents the restoration parameters of the 3D object. For the 2D boundary center [x... 2d ,y 2d ] and 3D projection center [x p ,yp The same anchoring center was applied.

[0103] This invention was tested on the KITTI autonomous driving dataset, a dataset jointly sponsored by the Karlsruhe Institute of Technology and Toyota Technological University at Chicago for research in the field of autonomous driving. The authors collected six hours of real-world traffic environment data, comprising multiple modalities including calibrated and synchronized images, radar scans, high-precision GPS information, and IMU acceleration information. The KITTI dataset contains 7481 images for training and 7518 images for testing. Since the ground truth values ​​for the test set have not been officially released, the experimental results on the test set were obtained by submitting the method proposed in this invention to the official KITTI website. Following other literature, this invention divides the training samples into a training set (3712 images) and a validation set (3769 images).

[0104] The model was trained, validated, and tested on the aforementioned dataset. It detected cars, pedestrians, and cyclists in a scene based on a single input image and output 3D bounding boxes. Results showed that in the validation set, when detecting cars with IoU=0.7, the average 3D accuracy was 29.70%, 20.64%, and 17.05% for easy, medium, and hard settings, respectively. In the test set, when detecting cars with IoU=0.7, the average 3D accuracy was 24.27%, 17.06%, and 14.76% for easy, medium, and hard settings, respectively; when detecting pedestrians with IoU=0.5, the average 3D accuracy was 13.30%, 8.25%, and 7.38% for easy, medium, and hard settings, respectively; and when detecting cyclists with IoU=0.5, the average 3D accuracy was 10.67%, 6.47%, and 5.62% for easy, medium, and hard settings, respectively. This result demonstrates higher accuracy compared to other detection models, indicating that the proposed model can learn and accurately detect targets of different categories. The detection results of the proposed method on the KITTI dataset for car targets are visualized as follows. Figure 2 As shown in the figure. The results show that the model of the present invention can accurately detect the target object, and the detection results are close to the actual values, demonstrating excellent performance.

Claims

1. A monocular 3D target detection method based on convolutional attention and feature decoupling, comprising the following steps: Step 1: Given an input image with a resolution of H×W, the backbone network outputs a feature map F. Using F as the original feature map, a series of convolutional layers are used to generate visual features f. V =Conv(F), and then implicitly learn deep features f using a depth estimator under the supervision of the depth map. D The method is as follows: For the original feature F, two convolutional layers are used to predict the probability of a discrete depth interval D. This probability represents the confidence level that the depth value of each pixel belongs to a certain depth interval. Linearly increasing discretization is used to discretize the true depth value from the continuous space to the discrete space d. i , i is the depth range index, d i This represents the i-th discrete interval, counted from shallowest to deepest. Let the intermediate feature map X = Conv(F) represent the initial depth-aware features. The feature center of each depth interval, i.e., the depth prototype, is calculated by aggregating the depth-aware features of each pixel belonging to the specified interval. Group convolution is used to generate the predicted depth intervals D. The number of intervals is reduced from N to N' = N / r using a set scale r. The depth prototype F d The feature map X' is generated by weighting the features X' of all pixels according to their respective probabilities to the depth category m. Let X' represent the feature of the i-th depth interval pixel in X' = Conv(X), and L be the pixel set in the feature map X'. Based on the prototype F d The reconstructed new deep features are denoted as f. D ; Step two involves inputting visual and depth features into a visual multi-scale detail convolutional encoder and a depth multi-scale detail convolutional encoder, respectively. The multi-scale detail convolutional encoder integrates a multi-scale asymmetric convolutional attention module to aggregate local features of the input image and encodes the aggregated local features, as follows: The input visual and depth features are each processed through a multi-scale asymmetric convolutional attention module, resulting in the attention map and output sum f of the multi-scale asymmetric convolutional attention module. out ; In the multi-scale asymmetric convolutional attention module, the input features are first aggregated by asymmetric convolution to aggregate local information; then multi-branch asymmetric convolution is used to capture multi-scale contextual information, and then 1×1 convolution is used to model the channel dimension. The output f of the multi-scale asymmetric convolutional attention module out The final output f of the multi-scale detail convolutional encoder is generated through a feedforward network and a batch normalization layer. c ; Step 3: The spatial and visual information of the input image is encoded and decoded using the dual encoder-dual decoder structure of the decoupled feature-guided self-attention model. The decoupled feature-guided self-attention model has two branches: depth and vision. Both branches use the same structure, and their parameters are trained separately. The two branches of the decoupled feature-guided self-attention model are respectively fed by the output of the depth multi-scale detail convolutional encoder. and the output of the visual multi-scale detail convolutional encoder As input; Step four: Use the cross-attention-guided fusion module to process the output F of the deep branch. D and the output F of the visual branch V Perform fusion; in the cross-attention-guided fusion module, F D As input query Q FD =Linear(F D ), F V As K FV V FV =Linear(F V The input is fed into a cross-attention layer; the output of the cross-attention layer is then passed through a layer normalization layer to obtain the final output F of the fusion module. fused : Step 5: A single-stage detector with predefined 2D-3D anchor points is used to regress the bounding box. Each predefined anchor point is represented by a 2D bounding box [x...]. 2d ,y 2d ,w 2d ,h 2d ] and 3D bounding box [x p ,y p ,z,w 3d ,h 3d ,l 3d The parameter composition of [x,θ], [x 2d ,y 2d ] and [x p ,y p [w] represents the center of the two-dimensional bounding box and the center of the three-dimensional object projected onto the image plane. 2d ,h 2d ] and [w 3d ,h 3d ,l 3d ] represents the physical dimensions of the 2D and 3D bounding boxes, respectively; z represents the depth of the 3D object center; θ represents the observation angle. During training, all ground truth values ​​are projected onto the 2D space to calculate the intersection of all 2D anchor points. Anchor points with an intersection greater than 0.5 are selected, and the corresponding 3D boxes are optimized accordingly. Step 6: Predict the 2D bounding box for each anchor point [t] x ,t y ,t w ,t h ] 2d and 3D bounding box [t x ,t y ,t w ,t h ,t l ,t z ,t θ ] 3d It parameterizes the residual values ​​of 2D and 3D bounding boxes, predicts classification scores, and outputs the recovered bounding boxes based on anchor points and network predictions.

2. The monocular three-dimensional target detection method according to claim 1, characterized in that, In step one, linearly increasing discretization is used to discretize the depth truth from the continuous space to the discrete space d. i : Where N is the number of depth intervals, [d min d max ] represents the depth range; pixels with depth values ​​outside this range will be marked as invalid and will not be used for optimization during training; i is the depth interval index, d i This represents the i-th interval, counted from shallowest to deepest.

3. The monocular three-dimensional target detection method according to claim 2, characterized in that, In step one, the number of depth intervals N is set to 96, and the depth range [d min d max The value is set to [1, 80], and r = 2.

4. The monocular three-dimensional target detection method according to claim 1, characterized in that, In step two, the asymmetric convolution is represented as: Where BN represents the batch normalization operation, γ i and β i These are learnable parameters in the BN operation, i = 1, 2, 3.

5. The monocular three-dimensional target detection method according to claim 1, characterized in that, The multi-scale asymmetric convolutional attention module mentioned in step two is as follows: f out =Attention×f Where f represents the input feature f D or f V Asy_Conv represents asymmetric convolution; Scale i Let i∈{0,1,2,3} represent the i-th branch; Attention and f out These represent the attention map and output of the multi-scale asymmetric convolutional attention module, respectively.

6. The monocular three-dimensional target detection method according to claim 1, characterized in that, In step three, for the depth branch, the method is as follows: 1) The output of the depth multi-scale detail convolutional encoder As input features, they are fed into deep branches of the self-attention model guided by decoupled features; 2) In the encoder of the depth branch, Encoding is performed using a self-attention layer and a layer normalization layer: Where Linear represents linear transformation, softmax is an activation function, C represents the dimension of the input features, LN represents layer normalization operation, and A represents the attention score. This represents the encoder output in the depth branch; 3) In the decoder of the deep branch, As input query Position code P as K P V P =Linear(P) is input into a cross-attention layer; the output of the cross-attention layer is decoded through a self-attention layer and a layer normalization layer to obtain the output F of the depth branch. D The implementation method of the cross-attention layer is as follows: Among them, A D This represents the attention score of the cross-attention layer; For the visual branch, process it in the same way to obtain the output F of the visual branch. V .

7. The monocular three-dimensional target detection method according to claim 1, characterized in that, In step six, the output bounding box is restored as follows: Where (^) represents the restoration parameter of the three-dimensional object; for the two-dimensional boundary center [x 2d ,y 2d ] and 3D projection center [x p ,y p The same anchoring center was applied.

Images