Monocular 3d object detection method based on instance-level adaptive depth estimation

Abstract

The application belongs to the field of computer vision, and provides a monocular three-dimensional target detection algorithm based on instance-level adaptive depth estimation, which comprises the following steps: firstly, using a DLA-34 backbone network trained from ImageNet as an encoder to encode initial multi-level features from an input image; then, using a global feature representation, combining a two-dimensional target detection region distribution to obtain an instance-level sparse depth map, and obtaining implicit depth space clue feature information by taking a predicted final depth value as a linear combination of discrete depth interval center values and instance feature probability representation; thereafter, introducing the depth clue feature information into a three-dimensional target detection branch by using a spatial enhancement attention module, and learning target space feature representations at different scales by a pyramid pooling operation; finally, using multiple loss functions to hierarchically optimize the supervision learning of multiple detection tasks, and training the model by using a hybrid loss function.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and autonomous driving, and in particular to a monocular three-dimensional object detection algorithm based on instance-level adaptive depth estimation. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Three-dimensional object detection is an important computer vision task that extracts key three-dimensional information such as object location, size, and orientation from images. Currently, many methods use sensors such as lidar, stereo cameras, and depth cameras to obtain data in the form of point clouds, stereo images, and depth images, effectively improving the accuracy of three-dimensional object detection. However, the cost and complexity of these sensors pose challenges to downstream tasks. In contrast, monocular three-dimensional object detection only requires two-dimensional images and camera calibration information to achieve three-dimensional object detection. Due to its simplicity and cost-effectiveness, monocular three-dimensional object detection has gained increasing attention in the fields of autonomous driving, visual navigation, and robotics.

[0004] Currently, there is a large performance gap between monocular-based methods and lidar-based methods. This can be attributed to the fact that lidar can provide more accurate and dense point cloud data, while monocular images produce limited and ambiguous depth cues. The inherent challenges mainly come from the limited spatial information present in monocular images, as well as the diversity of object sizes, shapes, and textures in different scenes. Therefore, the accuracy of monocular three-dimensional object detection is still subject to certain limitations. In fact, estimating the missing depth information from the input two-dimensional images has long been a major challenge in the field of computer vision.

[0005] Benefiting from the current research on monocular depth estimation and 3D object detection methods based on lidar technology, many researchers are turning their efforts to the field of monocular 3D object detection. Certain methods are based on the idea of extracting objects from the image plane and lifting 2D detection to 3D space. The simplest and most direct method is to utilize a pre-trained monocular depth estimation model to predict a dense depth map from an RGB image, then reconstruct a pseudo point cloud using pre-calibrated internal camera parameters, and finally implement detection through a laser-based 3D object detection framework. The above methods have the same problem, i.e. limited performance due to inaccurate depth prediction, lack of RGB context information, and inherent differences between real and pseudo LIDAR data. In addition, certain other research methods take depth estimation as an auxiliary task, guiding monocular 3D object detection models to learn depth-aware features, thereby achieving significant improvements in model inference speed and accuracy. However, these aforementioned methods are inevitably susceptible to depth estimation errors, resulting in feature ambiguity and suboptimal predictions.

[0006] Considering the practical requirements of downstream tasks, models need to meet the constraints of real-time and lightweight processing. Certain research methods add an additional task branch in the two-dimensional object detection framework to learn the three-dimensional spatial information of the detected objects. This is achieved by predicting the center position of the detected object (usually by selecting the center projection coordinates of the three-dimensional bounding box to the image plane) and the depth information and other three-dimensional attributes associated with the center position. However, some center-based methods heavily rely on direct regression depth precision, which constitutes a bottleneck for improving the performance of monocular 3D object detection. In order to fully utilize prior knowledge, some methods address depth by constructing more key point information and introduce more spatial geometric constraints. However, the occlusion and truncation of visible key points often lead to performance degradation, depending on the number and quality of available constraints, as these factors greatly affect accuracy. In addition, the 3D bounding box vertices projected in the 2D image have no explicit semantic meaning, making it difficult to accurately locate these vertices. Furthermore, relying solely on the internal key points of the object will lose geometric clues corresponding to other objects or the environment. In addition, considering the limitations of local spatial information prediction, this method does not take into account the global semantic spatial location information of the target. SUMMARY

[0007] To solve the above problems, we propose a monocular 3D object detection framework based on instance-level adaptive depth estimation. This framework introduces a new depth solving method that combines classification and regression for monocular 3D object detection tasks. First, based on the global context information of the scene level, we adaptively estimate the depth distribution of each scene to obtain the optimal depth discrete distribution of different image scenes. Finally, we linearly combine it with the probability distribution representation of high-level local object semantic features, fully exploring the spatial correlation between global and local clues. We design an adaptive box-width estimation (ABWE) module to solve this problem, which learns adaptive depth distribution boxes at multiple different scales in the global spatial context through three different dilation rates of dilated convolution. To focus on instance-level feature information, we use RoIAlign to extract local object probability representation and linearly combine the box centers. In addition, to improve the 3D perception ability of the model, we design a new spatial perception enhancement attention (SPEA) module. Specifically, we fuse spatial attention maps of different scales through a pyramid pooling layer to enhance the multi-scale spatial perception ability of the model and implicitly introduce depth clues to improve the depth feature representation ability of the model.

[0008] The present project provides a monocular 3D object detection algorithm based on instance-level adaptive depth estimation, which includes the following steps:

[0009] 1. As shown in Figure 1 , this monocular 3D object detection algorithm uses a DLA-34 backbone network trained from ImageNet as an encoder to encode initial multi-level features from input images;

[0010] 1.1) Before starting, we collect open-source datasets related to the field of image 3D object detection, including KITTI dataset, ScanNetV2 dataset, Waymo dataset, SUN RGB-D dataset, and Lyft L5 dataset.

[0011] 1.2) This invention, using the KITTI dataset with 80256 target markers to train the dataset for training the model; using the test dataset in the KITTI dataset for detecting the generalization performance of the model. In addition, we take the same test method as before, split the training set into 3712 training sets and 3349 validation sets for local test to verify the effectiveness of our model. In addition, we also use a series of data enhancement operations such as normalization, random horizontal inversion, random horizontal movement, random zooming and cropping in the data enhancement part, which effectively improve the detection performance of the model and the generalization performance of the model. Because the resolution of the picture sample in the dataset is inconsistent, we uniformly fill the picture to the resolution of (1280x342) in the subsequent network training.

[0012] 1.3) After collecting and sorting, first we use the pre-trained DLA-34 backbone network to extract the resolution of (1280x342) RGB picture, here in order to improve the calculation efficiency of the algorithm, we remove the deformable convolution part in the network. Extract multi-layer features from the backbone network, and send these features to the Neck network part, in order to improve the local and global feature representation of the model, we extract two different scale features respectively.

[0013] 2. As shown in Figure 2 , using global feature representation, the predicted final depth value is a linear combination of discrete depth interval center value and instance feature probability representation. First, adaptively generate adaptive box width. Then, generate instance-level probability distribution for local RoI region. Finally, linear combination is used to calculate the final depth value;

[0014] 2.1) First, select the 4th layer depth feature map of the backbone network as input. Then, in order to gradually expand the receptive field and enhance the network's feature extraction ability from local to global, three asymmetric convolution (hole convolution) layers with different expansion rates are used to extract three groups of context features with the same size. Then, the context information is integrated by element-level step-by-step addition, which realizes the fusion of these features with different scales. Then, the channel attention module is used to further enhance the features in the fusion area and enhance the information in the key area while suppressing the interference in the unimportant area. Finally, this module produces two outputs: one is the feature map after spatial attention processing, and the other is the n-dimensional vector b obtained by applying global average pooling operation to the fused feature map. Normalize the vector b and sum it to 1 to get the box width vector b, as shown below:

[0015]

[0016] where ∈ = 10 -3 Ensure that the width of each box is strictly positive.

[0017] 2.2) The fifth layer deep feature map of DLA34 is used as input, which represents high resolution and local pixel-level information. It is input into a 3x3 convolutional layer and element-wise added with the feature map of the adaptive box width estimation module to better complement global and fine-grained information. Then, the feature channels are added to N through a 1x1 convolutional layer, which needs to be consistent with the dimension of the vector b. In order to guide the model to focus on local object features, we use RoIAlign to crop and adjust the size of the RoI feature. The RoI feature only contains object-level features and does not include background noise. Finally, we pool the RoI feature with a size of 7x7xN for each object through global average pooling (GAP), and calculate a P∈R 1x1xN probability distribution through Softmax.

[0018] 2.3) Finally, given the predicted box length b in the adaptive box width estimation module, it first converts them to box centers through a simple post-processing process, as shown in the following formula:

[0019]

[0020] where c(b i ) is the center depth of the i th th box. d max and d min min are the maximum value and the minimum valid depth value of the dataset. Finally, the final depth value is obtained by linear combination as follows:

[0021]

[0022] 3. As shown in Figure 3 , the spatial enhanced attention module is used to introduce depth clue feature information into the three-dimensional target detection branch, and the pyramid pooling operation is used to learn the target spatial feature representation at different scales, improve the spatial perception ability of the model, and better capture the three-dimensional attribute information of the target;

[0023] 3.1) First, two convolutional layers with 1x1 filters are applied on the feature X to generate two new feature maps Query (Q) and Key (K), where {Q, K} is the new feature map with reduced channel number. Then Q and K are reshaped to R C′×N , where N=HxW. At the same time, another convolutional layer also uses 1x1 filters to connect with the Y feature map to generate Value (V).

[0024] 3.2) Use the multi-scale sampling of the pyramid and apply the pyramid pooling layer with kernel size {8, 6, 3, 2} on feature maps K and V. Through the fusion module, we divide the feature maps Q, K and V after pooling into n blocks in the channel dimension, and each block utilizes the information of different channels, so that each attention head can learn different information in the attention map. Finally, we apply a SoftMax layer on the feature S to calculate the spatial attention map a = N x N'. Then I apply the SoftMax layer to S to calculate the spatial attention map. Next, perform matrix multiplication on the blocks A and the transposed V after merging. Connect all the results and reshape them into R H×W×C .3.3) Repeat all the processes on K and V at different scales and sum them up to get the result M. Finally, collect the context information by element-level addition operation X and the sum of the results from multiple scales. Specifically as follows:

[0025] O i = γM i + X i , (4)

[0026] where i is each position in X, and γ is a learnable scale parameter. It is initialized to 0 and gradually learns to assign more weight.

[0027] 4. Use multiple joint loss functions to optimize multiple task supervised learning, mainly divided into two-dimensional target detection task loss and three-dimensional target detection task loss;

[0028] 4.1) The two-dimensional target detection task loss function contains: the focal loss function (L heatmap ) of the key point heat map of the object center position, the two-dimensional detection box size loss function (L size_2d ) and the two-dimensional center offset loss function (L offset_2d ). Specifically as follows:

[0029] L 2d = L heatmap + L size_2d + L offset_2d , (5)

[0030] 4.2) The three-dimensional target detection task loss function contains: the three-dimensional center offset loss function (L offset_3d ), the three-dimensional detection box loss function (L dimension ), the yaw angle loss function (L head ) and the depth estimation loss function (L depth ), specifically as follows:

[0031] L 3d = L head + L dimension + L offset_3d+L depth , (6)

[0032] where the depth loss function is given by:

[0033]

[0034] 4.3) The loss function of the two tasks is combined as follows:

[0035] L total = L 2d + L 3d , (8)

[0036] The advantages of the present application are that it alleviates the current monocular 3D object detection framework based on center-guided, which relies heavily on direct regression depth accuracy, which constitutes a bottleneck for improving the performance of monocular 3D object detection. In addition, the 3D bounding box vertices projected in the 2D image have no explicit semantic meaning, which makes it difficult to accurately locate these vertices. In addition, relying only on the internal key points of the object will lose the geometric clues corresponding to other objects or the environment. We introduce a new depth solving method that combines classification and regression for the monocular 3D object detection task. First, based on the global context information of the scene, we adaptively estimate the depth distribution of each scene to obtain the best depth discrete distribution of different image scenes. Finally, we linearly combine it with the probability distribution representation of the advanced local object semantic features, fully exploring the spatial correlation between global and local clues. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 Monocular 3D object detection flowchart of instance-level adaptive depth estimation

[0038] Figure 2 Adaptive instance-level depth width estimation module

[0039] Figure 3 Spatial perception enhanced attention module

[0040] Figure 4 Algorithm quantitative comparison table of "car" category

[0041] Figure 5 Algorithm quantitative comparison table of "pedestrian" and "bicyclist" categories

[0042] Figure 6 Qualitative comparison chart of validation set algorithm

[0043] Figure 7 Qualitative comparison chart of test set algorithm DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the inventive example of the monocular three-dimensional target detection algorithm based on instance-level adaptive depth estimation, and in addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the ordinary skilled in the art without creative labor are within the protection scope of the present application.

[0045] The flowchart framework of the present application is shown as Figure 1 The monocular three-dimensional target detection algorithm based on instance-level adaptive depth estimation of the present application is described as follows:

[0046] 1. As shown in Figure 1 The monocular three-dimensional target detection algorithm uses the DLA-34 backbone network trained from the ImageNet as an encoder to encode the initial multi-level features from the input image.

[0047] The DLA-34 backbone network is used as an encoder to encode the initial multi-level features from the input image.

[0048] 1.1) Before starting, we collect the open source datasets related to the field of image three-dimensional target detection, including the KITTI dataset, the ScanNetV2 dataset, the Waymo dataset, the SUN RGB-D dataset and the Lyft L5 dataset.

[0049] 1.2) In this invention, the KITTI dataset with 80256 target labels is used to train the dataset for training the model; the test dataset in the KITTI dataset is used to detect the generalization performance of the model. In addition, we use the same test method as before to split the training set into 3712 training sets and 3349 validation sets for local testing to verify the effectiveness of the model. In addition, we also use a series of data enhancement operations such as normalization, random horizontal inversion, random horizontal movement, random enlargement and cropping in the data enhancement part, which effectively improve the detection performance and model generalization performance of the model. Since the resolution of the picture samples in the dataset is inconsistent, we uniformly fill the picture to the resolution of (1280x342) in the subsequent network training.

[0050] 1.3) After the collection and arrangement are completed, first, we use the pre-trained DLA-34 backbone network to extract the RGB picture with a resolution of (1280x342). Here, in order to improve the calculation efficiency of the algorithm, we remove the deformable convolution part in the network. Multi-level features are extracted from the backbone network, and these features are sent to the Neck network part. In order to improve the local and global feature representation of the model, we extract two different scales of features respectively.

[0051] 2. As shown in Figure 2As shown, with the global feature representation, the predicted final depth value is taken as a linear combination of the discrete depth bin center values and the instance feature probability representation. First, adaptive box widths are generated adaptively. Then, instance-level probability distributions are generated for the local RoI region. Finally, the final depth value is calculated by linear combination.

[0052] 2.1) First, the feature maps at the 4th layer depth of the backbone network are selected as input. Subsequently, to gradually expand the receptive field and enhance the network's feature extraction ability from local to global, three groups of context features of the same size are extracted using three asymmetric convolution (dilated convolution) layers with different dilation rates. Then, the context information is integrated by element-level step-by-step addition, realizing the fusion of these features at different scales. Subsequently, the features in the fusion area are further enhanced by the channel attention module, enhancing the information of key areas while suppressing the interference of unimportant areas. Finally, this module produces two outputs: one is the feature map after spatial attention processing, and the other is an n-dimensional vector b obtained by applying a global average pooling operation to the fused feature map. The vector b is normalized and summed to 1 to obtain the box width vector b, as shown in the following formula:

[0053]

[0054] where ∈ = 10 -3 ensures that the width of each box is strictly positive.

[0055] 2.2) Using the feature maps at the 5th layer depth of the DLA-34 backbone network as input, this feature representation has high resolution and local pixel-level information. It is input into a 3x3 convolution layer and element-wise added with the feature map of the adaptive box width estimation module to better supplement global and fine-grained information. Then, through a 1x1 convolution layer, the feature channels are added to N, which needs to be consistent with the dimension of the vector b. In order to guide the model to focus on local object features, we use RoIAlign to crop and adjust the size of the RoI feature. The RoI feature only contains object-level features and does not include background noise. Finally, we pool the RoI feature of size 7x7xN for each object through global average pooling (GAP), and calculate a P∈R 1x1xN probability distribution through Softmax.

[0056] 2.3) Finally, given the predicted box lengths b in the adaptive box width test module, it first converts them to box centers through a simple post-processing process, as shown in the following formula:

[0057]

[0058] where c(b i ) is i thThe center depth of the box, d max and d min The minimum value is the maximum value and the minimum valid depth value of the data set. Finally, the final depth value is obtained by linear combination as follows:

[0059]

[0060] 3. As Figure 3 shown, the depth clue feature information is introduced into the three-dimensional target detection branch by using the spatial enhanced attention module, and the target spatial feature representation under different scales is learned through the pyramid pooling operation, so as to improve the spatial perception ability of the model and better capture the three-dimensional attribute information of the target.

[0061] 3.1) First, two convolutional layers with 1x1 filters are applied on the feature X to generate two new feature maps Query (Q) and Key (K), where {Q, K} is the new feature map with reduced channel number. Then Q and K are reshaped to R C′×N where N = H x W. At the same time, another convolutional layer also uses 1x1 filters to connect with Y feature map to generate Value (V).

[0062] 3.2) Multi-scale sampling is used by pyramid, and pyramid pooling layers with kernel size {8, 6, 3, 2} are applied on feature maps K and V. Through the fusion module, we divide the feature maps Q, K and V after pooling into n blocks in the channel dimension, and each block uses different channel information, so that each attention head can learn different information in the attention map. Finally, we apply a SoftMax layer on the feature S to calculate the spatial attention map a = N x N'. Then I apply the SoftMax layer on S to calculate the spatial attention map. Next, I perform matrix multiplication on the transpose of block A and the merged V. Connect all the results and reshape them to R H×W×C . 3.3) Repeat all the processes on K and V at different scales and sum them up to get the result M. Finally, collect the context information by element-level addition operation X and the sum result from multiple scales. Specifically, as shown in the following formula:

[0063] O i = γM i + X i , (4)

[0064] where i is each position in X, and γ is a learnable scale parameter. It is initialized to 0 and gradually learns to assign more weight.

[0065] 4. Multiple joint loss functions are used to optimize multiple task supervised learning, mainly divided into two-dimensional target detection task loss and three-dimensional target detection task loss.

[0066] 4.1) The loss function of the two-dimensional target detection task includes: the focal loss function (L heatmap ) of the key point heat map of the object center position, the two-dimensional detection box size loss function (L size_2d ), and the two-dimensional center offset loss function (L offset_2d ). Specifically, as shown in the following formula:

[0067] L 2d = L heatmap + L size_2d + L offset_2d , (5)

[0068] 4.2) The loss function of the three-dimensional target detection task includes: the three-dimensional center offset loss function (L offset_3d ), the three-dimensional detection box loss function (L dimension ), the yaw angle loss function (L head ), and the depth estimation loss function (L depth ). Specifically, as shown in the following formula:

[0069] L 3d = L head + L dimension + L offset_3d + L depth , (6)

[0070] wherein the depth loss function is as shown in the following formula:

[0071]

[0072] 4.3) The loss function of the joint two tasks is specifically as shown in the following formula:

[0073] L total = L 2d + L 3d , (8)

[0074] The above is the preferred implementation of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A monocular 3D target detection method based on instance-level adaptive depth estimation, characterized in that the method includes the following steps: 1.1) The DLA-34 backbone network trained from ImageNet is used as an encoder to encode initial multi-level features from the input image; 1.2) Utilizing global feature representation, the predicted final depth value is used as a linear combination of the discrete depth interval center value and the instance feature probability representation. Combined with the two-dimensional target detection region distribution, an instance-level sparse depth map is obtained, and implicit depth spatial cue feature information is derived. The specific method of step 1.2) is as follows: 3.1) First, the feature map at the 4th layer of the backbone network is selected as input. Then, to gradually expand the receptive field and enhance the network's feature extraction capability from local to global perspectives, three asymmetric convolutional layers with different dilation rates are used to extract three sets of contextual features of the same size. Next, the contextual information is integrated through element-wise summation, achieving the fusion of these features with different scales. A channel attention module is used to further enhance the features of the fusion region, strengthening the information in key regions while suppressing interference from unimportant regions. Finally, this module produces two feature outputs: one is the feature map after spatial attention processing, and the other is an n-dimensional vector b obtained by applying global average pooling to the fused feature map. Vector b is normalized and summed to 1 to obtain the bin width vector b, as shown in the following equation: Where ∈=10 -3 This ensures that the width of each box is a strictly positive number; 3.2) The feature map from the 5th layer of the DLA-34 backbone network is used as input, representing high-resolution and local pixel-level information. It is then fed into a 3×3 convolutional layer, and element-wise addition is performed using the feature map from the adaptive bin width estimation module to better supplement global and fine-grained information. Next, a 1×1 convolutional layer is used to add the feature channels to N, which needs to be consistent with the dimension of the vector b. To guide the model to focus on local object features, RoIAlign is used to crop and resize the RoI features. The RoI features only contain object-level features and do not include background noise. Finally, global average pooling (GAP) is used to pool each object into a 7×7×N RoI feature, and a P∈R is calculated using Softmax. 1x1xN Probability distribution; 3.3) Finally, given the predicted box lengths b in the adaptive box width test module, it first converts them into box centers through a simple post-processing procedure, as shown in the following formula: Where c(b) i) is i th The center depth of the box, d max and d min The minimum value is the minimum effective depth value of the dataset, which is then obtained by linearly combining the maximum value and the minimum effective depth value of the dataset. Finally, the final depth value is obtained by the following formula: 1.3) The spatial augmentation attention module is used to introduce depth cue feature information into the 3D target detection branch, and the spatial feature representation of the target at different scales is learned through pyramid pooling operations to improve the spatial perception capability of the model. The fused features are then used as the feature input for the detection head. The specific method of step 1.3) is as follows: 4.1) First, two convolutional layers with 1×1 filters are applied to feature X to generate two new feature maps Query(Q) and Key(K), where {Q, K} is the reduced number of channels of the new feature maps; then Q and K are reshaped into R in their spatial dimensions. C′×N , where N = H × W; at the same time, another convolutional layer also uses a 1 × 1 filter to connect to the Y feature map to generate Value(V); 4.2) Utilize multi-scale sampling of the pyramid and apply pyramid pooling layers with kernel sizes of {8,6,3,2} to feature maps K and V; Through a fusion module, the feature map Q, the pooled K, and V are divided into n blocks along the channel dimension. Each block utilizes information from different channels, allowing each attention head to learn different information from the attention map. Finally, a SoftMax layer is applied to feature S to compute the spatial attention map a = N × N'. Then, a SoftMax layer is applied to S to compute the spatial attention map. Next, matrix multiplication is performed on the transpose of block A and the merged V. All results are concatenated and reshaped into R. H ×W×C ; 4.3) Repeat all processes and sum them at different scales of K and V to obtain the result M; finally, collect contextual information through element-wise addition operations X and summations from multiple scales; as shown in the following equation: OR i =γM i +X i , (4) Where i is each position in X, and γ is a learnable scale parameter; it is initialized to 0 and gradually learns to assign more weights. 1.4) Supervised learning for multiple detection tasks is optimized hierarchically using multiple loss functions, which are divided into two-dimensional target detection task loss and three-dimensional target detection task loss, and the model is trained using a hybrid loss function.

2. The monocular 3D target detection method based on instance-level adaptive depth estimation according to claim 1, characterized in that: The specific method for step 1.1) is as follows: 2.1) Before starting, collect relevant open-source datasets in the field of 3D object detection, including the KITTI dataset, ScanNetV2 dataset, Waymo dataset, SUN RGB-D dataset, and Lyft L5 dataset; 2.2) The KITTI dataset with 80,256 target labels was used to train the model. The test dataset from the KITTI dataset was used to test the model's generalization performance. In addition, the same testing method as before was adopted, splitting the training set into 3,712 training sets and 3,349 validation sets for local testing to verify the model's effectiveness. Furthermore, a series of data augmentation operations were used in the data augmentation part, including normalization, random horizontal flipping, random horizontal shifting, and random scaling and cropping. These operations effectively improved the model's detection performance and generalization performance. Due to the inconsistency in the resolution of the image samples in the dataset, the images were uniformly filled to a resolution of 1280x342 in subsequent network training. 2.3) After the collection and organization are completed, the RGB images with a resolution of 1280x342 are first extracted using the pre-trained DLA-34 backbone network. Here, in order to improve the computational efficiency of the algorithm, the deformable convolution part in the network is removed. Multi-layer features are extracted from the backbone network and these features are fed into the Neck network part. In order to improve the local and global feature representation of the model, two different scales of features are extracted respectively.

3. The monocular 3D target detection method based on instance-level adaptive depth estimation according to claim 1, characterized in that: The specific method for step 1.4) is as follows: 5.1) The loss function for two-dimensional object detection tasks includes: the focal loss function L of the keypoint heatmap at the object's center location. heatmap Two-dimensional detection box size loss function L size_2d and the two-dimensional center offset loss function L offset_2d The specific formula is as follows: L 2d =L heatmap +L size_2d +L offset_2d (5) 5.2) The loss function for 3D object detection includes: 3D center offset loss function L offset_3d 3D detection box loss function L dimension Yaw angle loss function L head and depth estimation loss function L depth The specific formula is as follows: L 3d =L head +L dimension +L offset_3d +L depth (6) The depth loss function is shown in the following equation: 5.3) The loss function for both tasks is as follows: L total =L 2d +L 3d (8)

Images