Automatic driving scene three-dimensional target detection method and device based on spatio-temporal information fusion

Abstract

The application discloses an automatic driving scene three-dimensional target detection method based on space-time information fusion, which comprises the following steps: constructing a target detection network model with global feature prediction and local feature prediction; in the global feature prediction stage, a Transformer architecture is adopted to predict a 3D target category and correct an initialized 3D recommendation box; in the local feature prediction stage, the 3D recommendation box is mapped to a 2D space, local features in a current frame and a previous frame are extracted through a region of interest (ROI) pooling process, and the local features are input into an R-CNN structure to obtain final category prediction and position estimation. The application significantly improves the target detection accuracy in the automatic driving environment, optimizes the processing flow, and realizes efficient and accurate target recognition in a multi-view 3D environment. This not only improves the overall performance of the automatic driving system, but also provides a solid guarantee for safety in complex driving environments.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and specifically to a method for detecting 3D targets in multi-view autonomous driving scenarios based on spatiotemporal information fusion. Background Technology

[0002] 3D object detection is a key computer vision technology whose main purpose is to identify and locate objects, such as vehicles, pedestrians, or buildings, in three-dimensional space, while determining their spatial orientation. It has wide applications in multiple fields, including autonomous driving, robotics, video surveillance, and virtual and augmented reality.

[0003] The technological foundation of 3D object detection encompasses multiple fields, including computer vision, deep learning, and 3D graphics processing. Deep learning, especially convolutional neural networks (CNNs), has become a core technology for object detection. Despite significant advancements in deep learning, relying solely on visual information for 3D object detection still presents significant challenges. These challenges primarily include: first, the lack of depth information from single-frame images, i.e., precise localization in 3D space, makes locating the spatial position of the target from a single image extremely difficult; second, the underutilization of global and local features; and third, the effective utilization of spatiotemporal information is also a challenge in this task. Summary of the Invention

[0004] To improve the accuracy of 3D target detection and enhance the pose estimation capability of continuously moving targets, this invention designs an innovative 3D target detection method and system for multi-view autonomous driving scenarios. This method is based on spatiotemporal information fusion. This invention processes multiple consecutive video image sequences and outputs the 3D coordinates and category information of the target. The technical process consists of the following key steps:

[0005] The multi-view autonomous driving scene 3D target detection method based on spatiotemporal information fusion designed in this invention includes the following steps:

[0006] S1, preprocesses multiple consecutive video images;

[0007] S2, extract time-series features from the preprocessed data;

[0008] S3. Construct a target detection network model with global feature prediction and local feature prediction. In the global feature prediction stage, the Transformer architecture is used to predict the 3D target category and correct the initialized 3D recommendation box. In the local feature prediction stage, the 3D recommendation box is first mapped to 2D space. Local features in the current frame and the previous frame are extracted through the Region of Interest (ROI) pooling process. The local features are then input into the R-CNN structure to obtain the final category prediction and position estimation.

[0009] S4, train the object detection network model;

[0010] S5 uses a trained model for 3D object detection.

[0011] Furthermore, the preprocessing includes:

[0012] Identify and remove frames with abnormal values ​​that cause detection errors;

[0013] Normalization is performed on the mean and standard deviation of the RGB three channels;

[0014] The data is reordered and randomly shuffled.

[0015] Furthermore, in S2, a deep learning framework is used to extract core information from consecutive keyframes, and then multi-scale fusion is performed to output multi-scale feature vectors.

[0016] Furthermore, in the global feature prediction stage, the initialized Query, 3D recommendation boxes, and features from layer P4 of the Feature Pyramid Network (FPN) are used as inputs to the Transformer layer. Features from layer P4 are flattened and then used as Key and Value inputs to the Transformer module to capture the relationships between input features. Positional encoding is designed for the Query and Key, specifically including:

[0017] Query position encoding: The three coordinate dimensions (x, y, z) of the recommendation box are transformed by sin-cos position encoding. The transformed position encoding is then processed by a convolutional neural network (CNN) to obtain the final query position encoding.

[0018] Key position encoding: The absolute position of the image is processed by sin-cos position encoding, and then passed through a corresponding convolutional neural network to serve as the position encoding of the key.

[0019] Furthermore, the R-CNN structure comprises a self-attention mechanism, dynamic convolution, a feedforward network (FFN), and a predictive multilayer perceptron (MLP).

[0020] Furthermore, the process of mapping the 3D recommendation box to 2D space is as follows:

[0021] Initialize the 3D target bounding box;

[0022] Convert the initialized 3D target bounding box into a radar coordinate system representation;

[0023] Choose the camera with the best angle;

[0024] Using the selected camera number, find the vertices of the 3D recommendation box and calculate the two-dimensional coordinates of the vertices in the camera plane;

[0025] A 2D recommendation box is generated based on the projection of the vertex.

[0026] Furthermore, during model training, the Hungarian algorithm is used to establish a one-to-one correspondence between predicted values ​​and true values. The cost function of the Hungarian algorithm includes a class cost loss used to calculate the difference between the predicted probabilities of each anchor point belonging to different categories, and a box cost loss used to measure the difference between the predicted 3D bounding box and the true label.

[0027] Furthermore, the model's loss includes regression loss and classification loss. Regression loss measures the difference between the predicted bounding box and the actual bounding box, while classification loss addresses class imbalance.

[0028] Based on the same inventive concept, the present invention also discloses an electronic device, comprising:

[0029] One or more processors;

[0030] Storage device for storing one or more programs;

[0031] When one or more programs are executed by the one or more processors, the one or more processors implement a three-dimensional target detection method for autonomous driving scenarios based on spatiotemporal information fusion.

[0032] Based on the same inventive concept, the present invention also discloses a computer-readable medium storing a computer program, characterized in that: when the program is executed by a processor, it implements a three-dimensional target detection method for autonomous driving scenarios based on spatiotemporal information fusion.

[0033] The advantages of this invention lie in the development of an advanced spatiotemporal transformation multi-view network. This network effectively enhances its robustness through both global and local features. Furthermore, when utilizing local features, it cleverly tracks the 3D position of an object from the current frame to any frame in the history, effectively expanding the receptive range of relevant local features and thus improving detection capabilities. In addition, the proposed 3D-to-2D recommendation box conversion method effectively and accurately projects the 3D recommendation box in space onto the corresponding image, thereby obtaining the features of the ROI. Experimental results strongly demonstrate the superiority of our method compared to existing technologies.

[0034] An innovative multi-view 3D object detection method is particularly suitable for autonomous driving scenarios. This technique first utilizes a Transformer model to comprehensively learn global features and perform preliminary object prediction. Then, using a homogeneous matrix, these predictions are transferred to adjacent temporal keyframes, enhancing the coherence of spatiotemporal information. Furthermore, a 3D-to-2D feature transformation technique is proposed to effectively extract 2D ROI features from key regions in 3D space. Finally, R-CNN technology is used to deeply learn these local features, optimizing and correcting the initial predictions, thus improving the accuracy and reliability of detection.

[0035] A 3D-to-2D recommendation box conversion method is proposed to find the optimal viewing angle in a multi-camera driving environment. By analyzing the intrinsic and extrinsic parameters of the cameras, 3D recommendation boxes are projected onto images from multiple viewing angles, and the center point of the recommendation box with the smallest angle to the central axis of the camera is selected to determine the most suitable camera. Using the selected optimal camera image, ROI technology is employed to accurately extract local features, further improving the object detection performance.

[0036] This invention significantly improves target detection accuracy in autonomous driving environments, optimizes the processing flow, and achieves efficient and accurate target recognition in multi-view 3D environments. This not only enhances the overall performance of autonomous driving systems but also provides a solid guarantee for safety in complex driving environments.

[0037] Attached Figure Description Figure 1 This is a diagram of the overall architecture of a deep learning neural network according to an embodiment of the present invention.

[0038] Figure 2 This is the process of converting a 3D frame into a 2D frame according to the present invention. Detailed Implementation

[0039] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0040] Example 1

[0041] This invention designs a multi-view 3D target detection method for autonomous driving scenarios based on spatiotemporal information fusion. The overall architecture diagram of its deep learning neural network is attached. Figure 1 As shown, the specific implementation includes the following steps:

[0042] Step S1: Data Preprocessing. Detailed preprocessing operations are performed on images captured by cameras from six different perspectives around the vehicle. These include resizing, cropping to fit the model's input size requirements, and color space conversion to ensure consistency in format and characteristics across all images. First, abnormal frames causing detection errors are identified and removed. Second, RGB three-channel normalization is performed to prevent gradient explosion or vanishing gradients during training. Finally, to enhance the model's training efficiency and generalization ability, the data is reordered and randomly shuffled. Furthermore, preprocessing includes image enhancement techniques such as contrast adjustment and noise suppression to improve image quality and usability. The preprocessed images are then stitched together in a preset order to create a panoramic view, constructing a complete picture covering the vehicle's surrounding environment. This ensures the model can demonstrate superior performance and effectiveness in complex and dynamic traffic environments.

[0043] Step S2: Temporal Feature Extraction. Feature extraction achieves the goal of extracting multi-scale feature maps from the input image by combining a backbone network with a feature pyramid network (FPN). In this embodiment, the backbone network can include ResNet-50, VOV2-99, Swin Transformer, etc., and extracts core information from N consecutive frames (N>=2). Subsequently, the feature pyramid network (FPN) enhances the expressiveness of the features, generating multi-frame, multi-scale feature vectors containing rich semantic information, such as P2, P3, P4, and P5. The feature vectors capture key information of the target at different scales, providing necessary contextual support for subsequent target detection tasks and enhancing the model's ability to perceive and process dynamic changes in the target.

[0044] Step S3: Construct an object detection network model with global and local feature prediction. In the global feature prediction stage, a Transformer architecture is used to predict the 3D object category and correct the initialized 3D recommendation boxes. In the local feature prediction stage, the 3D recommendation boxes are first mapped to 2D space. Local features from the current and previous frames are extracted through Region of Interest (ROI) pooling. These local features are then input into an R-CNN structure to obtain the final category prediction and location estimate. The process of converting 3D boxes to 2D boxes is shown in the attached figure. Figure 2 As shown.

[0045] To fully leverage the Transformer's strengths in processing global image features and R-CNN's capabilities in processing local features, this model employs a highly efficient mechanism for processing temporal features and multi-camera data. This addresses the challenge of accurately predicting the spatial location of 3D targets in a single frame due to a lack of depth information. Traditional target detection methods often fail to provide sufficient depth information in a single frame, leading to significant errors in spatial location prediction. Therefore, this model introduces temporal features, utilizing image information from multiple time points to compensate for the limitations of a single frame, thus more accurately inferring the target's 3D spatial location. In this model, each frame's input image data comes from multiple cameras. Using all this temporal data simultaneously as the value for the Transformer layer would drastically reduce the Transformer's processing efficiency, creating a performance bottleneck. Therefore, the first Transformer layer in this model only processes the image data from the current time point. This maintains the Transformer's ability to perceive global features while reducing computational burden and improving model efficiency. The subsequent R-CNN layer processes image data from multiple time points and further optimizes 3D target localization and classification through the combination of temporal and local features. Specifically, the model operation is divided into three main stages: initialization stage, global feature prediction stage, and local feature prediction stage.

[0046] Model initialization phase: Initialize the 3D recommendation box and query value Query.

[0047] As a preferred approach, the model initialization phase initializes the 3D recommendation and query in a learnable manner, effectively addressing the complex environment and dynamic targets in autonomous driving. Specifically, this includes:

[0048] Recommendation box design and creation: Based on the requirements of object detection in 3D space, specific learnable recommendation boxes are designed. Each recommendation box has 10 dimensions, specifically including center coordinates (x, y, z), size (w, l, h), orientation (sin, cos), and velocity (vx, vy). These parameters are used to describe the geometric information and motion characteristics of the box in 3D space.

[0049] Recommendation box initialization: m*m (default m=30) recommendation boxes are generated using the nn.Embedding module. These recommendation boxes serve as the initial input to the model. To ensure that these recommendation boxes are uniformly distributed in 3D space, torch.meshgrid is used for uniform spatial arrangement.

[0050] Query initialization: Learnable queries are initialized, with a shape of [m*m, C], where C represents the number of feature channels. These queries are combined with recommendation boxes to interact with global and local features in subsequent stages, providing foundational information for object detection.

[0051] Global Feature Prediction Stage: Features from layer P4 of the Feature Pyramid Network (FPN) are fed into the Transformer module. This module combines self-attention, cross-attention, a feedforward network (FFN), and a normalization layer to effectively capture complex relationships between views. The self-attention mechanism enables the model to identify global correlations in the input features, while cross-attention helps establish deep connections between different input data, enhancing the expressive power of the features. The feedforward network further performs non-linear transformations on these features, enhancing the model's predictive ability. The Transformer's output is processed and predicted by a Multilayer Perceptron (MLP). Specifically, the MLP is responsible for converting the extracted global features into preliminary predictions of 3D object categories and correcting the position and size of the 3D recommendation boxes. The output of this stage provides foundational information for the subsequent R-CNN module.

[0052] A preferred approach is to improve target perception in autonomous driving through improved query and key location encoding, specifically including:

[0053] Query position encoding: The three coordinate dimensions (x, y, z) of the recommendation box are transformed using sin-cos position encoding. The transformed position encoding is then processed by a convolutional neural network (CNN) to obtain the final query position encoding. This encoding method ensures the model's ability to perceive the position of the recommendation box in three-dimensional space and provides accurate positional information for subsequent feature extraction and matching.

[0054] Key positional encoding: The absolute position of the image, i.e., the spatial position of the image pixels, is also processed through sin-cos positional encoding, and then passed through the corresponding convolutional neural network as the input to the Transformer for key positional encoding. This method ensures that the model accurately captures positional information in the image space, making spatial information and feature information closely integrated.

[0055] Local Feature Prediction: This stage utilizes five specially designed R-CNN layers. These R-CNN modules consist of a self-attention mechanism, dynamic convolution, a feedforward network (FFN), and a predictive multilayer perceptron (MLP), possessing powerful feature extraction and local optimization capabilities. First, based on the positional information of the 3D recommendation boxes predicted in the previous stage, the system infers the position of the 3D recommendation boxes at historical time step (t-1). Combining the 3D information from the previous and current frames, the model calculates the target's 3D position at the previous time step. Historical Time Step Position Calculation: Since single-frame images lack true depth information, directly mapping image pixels to 3D space may result in ghosting or inaccurate positioning. Therefore, the model calculates the target's spatial position over the previous N time steps and, combined with parameters such as the target's current position, velocity, and time, infers the target's 3D recommendation boxes at past time steps.

[0056] 3D-2D bounding box mapping: Using a projection method and a pose transformation matrix, the positions of these temporal 3D bounding boxes are mapped to the 2D plane in the image, ensuring a one-to-one correspondence between each 3D bounding box and a 2D bounding box in the image. The detailed process includes:

[0057] Representation of 3D recommendation boxes. Each box is represented as a tensor of (N, 10), where N represents the number of recommendation boxes.

[0058] Radar coordinate transformation of 3D target boxes. The tensor in (N,10) format is converted to a representation in radar coordinate system (N,8,3). Each 3D box is defined by the three-dimensional coordinates (x, y, z) of its eight vertices. The position of the center point of the 3D box is calculated.

[0059] Select the optimal camera angle. Considering the system is configured with multiple cameras, each fixed at different positions on the vehicle, covering different viewing angles (145°, 90°, 35°, -160°, -90°, -20°), calculate the angle between the center point and the central axes of these cameras, and select the 3D recommendation box to project onto the 2D camera encoding with the smallest angle.

[0060] Based on the appropriate camera parameters selected in the previous step, generate 2D recommendation boxes from the 3D bounding boxes. The specific steps are as follows:

[0061] 1) Calculate the coordinates of the 8 vertices of the 3D bounding box, which has the shape (N, 8, 3), where 8 represents the eight corner points of the 3D bounding box, and each corner point contains three-dimensional coordinates (x, y, z).

[0062] 2) Matrix multiplication operation: Perform matrix multiplication operation on the vertices of each 3D box with the corresponding camera calibration matrix to obtain the two-dimensional coordinates of each 3D vertex on the camera image plane.

[0063] 3) 2D Recommendation Box Generation: After obtaining the 2D coordinates of the vertices of the 3D bounding box on the corresponding camera image plane, the largest bounding area is selected to generate the final 2D recommendation box. The 3D proposal box is a 3D bounding box learned from the training data through a machine learning algorithm, which is then converted into a 2D recommendation box.

[0064] The above steps efficiently and accurately convert 3D target bounding boxes in the radar coordinate system into 2D recommended bounding boxes suitable for camera images, ensuring accurate target localization and identification in complex and ever-changing autonomous driving environments. This spatiotemporal information fusion-based processing flow greatly improves the performance and reliability of 3D target detection, providing solid technical support for the visual perception capabilities of autonomous driving systems.

[0065] After generating accurate 3D-2D recommendation box mappings, a Region of Interest (ROI) pooling process is used to extract local features of the target from historical image features. ROI-Align is performed on the acquired multi-frame 2D recommendation boxes to extract high-precision local features. Compared to ROI-Pooling, ROI-Align effectively avoids information loss through precise interpolation methods, although its computation is more complex. This method enables the model to more accurately capture and analyze key regions in the image, greatly improving the accuracy of target detection while reducing interference from background noise and irrelevant information, thus optimizing model performance and computational efficiency.

[0066] In local feature prediction, dynamic convolution exhibits excellent adaptability, dynamically adjusting the convolution kernel based on changes in input features to better capture details and variations in the target object. The structure of the local feature prediction stage includes not only dynamic convolution but also a multilayer perceptron (MLP). The MLP further processes the extracted features to predict the target's category and accurate 3D bounding box coordinates. To improve detection accuracy and robustness, this stage is repeated five times, each time continuously adjusting the position and category of the 3D bounding box to enhance the accuracy of target classification and 3D bounding box regression. This multi-iteration and feature optimization process effectively enhances target detection performance, ensuring higher classification accuracy and bounding box localization accuracy in 3D target detection tasks involving moving objects and complex scenes.

[0067] Step S4: Loss needs to be calculated between the predicted results and the true values ​​at each layer of the model. However, the number of recommendation boxes is far greater than the number of true boxes. Therefore, the Hungarian algorithm is used to establish a one-to-one correspondence between predicted and true values. First, a subset of predicted values ​​that match the true values ​​is selected from the prediction set for loss calculation. Through this algorithm, the model can find an optimal one-to-one match. Then, a specific cost function is used to calculate the matching cost between each predicted value and the true value. Specifically:

[0068] The class cost is used to calculate the difference between the predicted probabilities of each anchor point belonging to different classes. For the predicted probability p{i,j} of the i-th anchor point belonging to the j-th class, Focal Loss is used for calculation. Focal Loss effectively reduces the contribution of easily classified samples to the loss, thus focusing on more difficult samples and improving the model's training performance. The parameter γ is the focusing parameter, used to adjust the balance between easily and difficult-to-classify samples.

[0069] C Cost ( p i, j )=-(1- p i, j ) γ log ( p i, j )

[0070] The box cost measures the difference between the predicted 3D bounding box and the ground truth label. A smooth L1 loss is used for calculation, which is more stable when handling outliers. For the i-th anchor point, bi represents the predicted 3D bounding box, and bg,i represents the ground truth 3D bounding box. The box cost is obtained by calculating the smooth L1 distance between the predicted box and the ground truth box in each dimension.

[0071] B Cost ( b i , b g,i )=∑ x,y Smooth L 1( b i,j - b g,i,j )

[0072] Total cost T Cost Overall cost is a combination of class cost and box cost, used to measure the overall degree of matching between predicted and actual values. Cost The calculation method is the sum of class cost and box cost, i.e., T. Cost = C Cost (pi, j) + B Cost (bi,bg,i). The Hungarian algorithm can find the optimal correspondence between the predicted and true values, thereby minimizing the overall cost and optimizing the training effect of the model.

[0073] T Cost = C Cost ( p i, j ) + B Cost ( b i ,b g,i ).

[0074] By inputting the overall cost matrix into the Hungarian matching algorithm, a set of predictions can be selected to match the true values ​​one by one, providing a basis for subsequent loss calculation.

[0075] Step S5: Calculate the loss using the matching results obtained in Step S4. This loss function combines regression loss and classification loss. By considering the correspondence between predicted and true values, it enables the model to better learn the features and attributes required for object detection. The following is a detailed explanation of each part of the loss function: The regression loss measures the difference between the predicted bounding box and the actual bounding box. Specifically, to address the class imbalance problem, this method uses FocalLoss from the PyTorch library as the classification loss function. FocalLoss, by introducing a focusing parameter and the form of FocalLoss, makes the model focus more on hard-to-classify samples and reduce the contribution of easy-to-classify samples. This helps improve the model's performance on hard samples, thereby enhancing the model's training effect. This method uses L1Loss to calculate the regression loss. This loss function evaluates the consistency between the predicted bounding box and the true bounding box by measuring the smooth L1 distance between each dimension of the predicted bounding box and the true bounding box. It maintains stability when handling outliers, thus effectively guiding the model to learn the location and size information of the target.

[0076] Example 2

[0077] Based on the same inventive concept, the present invention also provides an electronic device, including one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in Embodiment 1.

[0078] Since the device described in Embodiment 2 of this invention is the electronic device used in implementing the three-dimensional target detection method for autonomous driving scenarios based on spatiotemporal information fusion in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this electronic device based on the method described in Embodiment 1 of this invention, and therefore will not be described again here. All electronic devices used in any method of this invention fall within the scope of protection of this invention.

[0079] Example 3

[0080] Based on the same inventive concept, the present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Embodiment 1.

[0081] Since the device described in Embodiment 3 of this invention is a computer-readable medium used to implement the three-dimensional target detection method for autonomous driving scenarios based on spatiotemporal information fusion in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this electronic device based on the method described in Embodiment 1 of this invention, and therefore will not be described again here. All electronic devices used in any method of this invention fall within the scope of protection of this invention.

Claims

1. A method for 3D target detection in autonomous driving scenarios based on spatiotemporal information fusion, characterized in that, Includes the following steps: S1, preprocesses multiple consecutive video images; S2, extract time-series features from the preprocessed data; S3. Construct a target detection network model with global feature prediction and local feature prediction. In the global feature prediction stage, the Transformer architecture is used to predict the 3D target category and correct the initialized 3D recommendation box. In the local feature prediction stage, the 3D recommendation box is first mapped to 2D space. Local features in the current frame and the previous frame are extracted through the Region of Interest (ROI) pooling process. The local features are then input into the R-CNN structure to obtain the final category prediction and position estimation. In the global feature prediction stage, the initialized Query, 3D recommendation box and features in the Feature Pyramid Network (FPN) are used as inputs to the Transformer layer. After flattening, the features are used as Key and Value inputs to the Transformer module to capture the correlation between input features. Design positional encoding for both Query and Key, specifically including: Query position encoding: The three coordinate dimensions (x, y, z) of the recommendation box are transformed by sin-cos position encoding. The transformed position encoding is then processed by a convolutional neural network (CNN) to obtain the final query position encoding. Key position encoding: The absolute position of the image is processed by sin-cos position encoding, and then passed through the corresponding convolutional neural network to serve as the position encoding of the key; S4, train the object detection network model; S5 uses a trained model for 3D object detection.

2. The method for 3D target detection in autonomous driving scenarios based on spatiotemporal information fusion according to claim 1, characterized in that: Preprocessing includes: Identify and remove frames with abnormal values ​​that cause detection errors; Normalization is performed on the mean and standard deviation of the RGB three channels; The data is reordered and randomly shuffled.

3. The method for 3D target detection in autonomous driving scenarios based on spatiotemporal information fusion according to claim 1, characterized in that: S2 uses a deep learning feature extraction network to extract core information from consecutive keyframes, and then performs multi-scale fusion to output multi-scale feature vectors.

4. The method for three-dimensional target detection in autonomous driving scenarios based on spatiotemporal information fusion according to claim 1, characterized in that: The R-CNN architecture consists of a self-attention mechanism, dynamic convolution, a feedforward network (FFN), and a predictive multilayer perceptron (MLP).

5. The method for three-dimensional target detection in autonomous driving scenarios based on spatiotemporal information fusion according to claim 1, characterized in that: The process of mapping a 3D recommendation box to 2D space is as follows: Initialize the 3D target bounding box; Convert the initialized 3D target bounding box into a radar coordinate system representation; Choose the camera with the best angle; Using the selected camera number, find the vertices of the 3D recommendation box and calculate the two-dimensional coordinates of the vertices in the camera plane; A 2D recommendation box is generated based on the projection of the vertex.

6. The method for three-dimensional target detection in autonomous driving scenarios based on spatiotemporal information fusion according to claim 1, characterized in that: During model training, the Hungarian algorithm is used to establish a one-to-one correspondence between predicted and true values. The cost function of the Hungarian algorithm includes class cost loss, which is used to calculate the difference between the predicted probabilities of each anchor point belonging to different categories, and box cost loss, which is used to measure the difference between the predicted 3D bounding box and the true label.

7. The method for three-dimensional target detection in autonomous driving scenarios based on spatiotemporal information fusion according to claim 1, characterized in that: The model's loss includes regression loss and classification loss. Regression loss measures the difference between the predicted bounding box and the actual bounding box, while classification loss addresses class imbalance.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images