Quick aerial view model perception method based on decoupling type depth supervision

Abstract

The invention discloses a quick aerial view model sensing method based on decoupling type depth supervision. The method comprises the following steps: acquiring a training data set; selecting a group of training data in the training data set as first training data; inputting the multi-view image into a feature extraction model to obtain an image feature and a depth feature; inputting the laser radar point cloud in the first training data into a depth-assisted algorithm to obtain a depth truth value graph and an edge weight graph; inputting the image features and the depth features into an aerial view detection model to obtain predicted object features; calculating to obtain a total loss function value and optimizing the aerial view detection model and the feature extraction model; selecting another group of training data in the training data set as new first training data, and repeatedly executing until the total loss function value reaches a preset condition; acquiring a multi-view image to be perceived; the method comprises the following steps: preprocessing a to-be-sensed multi-view image to obtain a preprocessed multi-view image; and inputting the preprocessed multi-view image into the trained feature extraction model and the trained aerial view detection model to obtain the features of the perceived object. The method is low in calculation complexity and high in sensing precision.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more specifically, to a fast bird's-eye view model perception method based on decoupled deep supervision. Background Technology

[0002] Currently, the improvement of AI hardware computing power and breakthroughs in deep learning algorithms are driving autonomous driving technology into a rapid development stage. At the commercial application level, mass-produced vehicles are gradually evolving from Level 2 assisted driving covering structured roads to Level 2+ and even higher-level assisted driving functions covering more complex urban scenarios. This evolution places far higher demands on the perception capabilities of autonomous vehicles than before: First, the perception field of view has expanded from the original forward-looking perspective to a global field of view. Second, the perception system not only needs to identify obstacles more accurately, but also needs to construct a unified, coherent, and easily usable scene representation for downstream tasks (such as prediction and planning). Against this backdrop, BEV (Bird's-Eye View) model perception has emerged and is gradually becoming the core paradigm of high-level autonomous driving perception systems. Compared with traditional forward-looking or perspective-based perception, the bird's-eye view integrates information from different sensors (especially multi-camera systems) at the feature level in a unified top-down coordinate system, effectively avoiding information loss and conflict problems caused by target-based result fusion. Meanwhile, the feature map from the BEV (bird's-eye view) perspective is naturally aligned with the vehicle coordinate system, which greatly simplifies the input processing flow of the subsequent trajectory prediction and motion planning modules, laying a solid foundation for building an end-to-end autonomous driving system, and also significantly saving the overall computing and memory overhead of the system.

[0003] From the perspective of the core mechanisms for converting 2D image features into 3D bird's-eye view features, existing mainstream technical solutions can be mainly divided into two categories: The first category is the query-interaction mechanism based on Transformer. This method performs cross-attention calculations between a set of learnable BEV query vectors and image features to aggregate and generate BEV features, such as BEVFormer and PETRv2. Although this type of method has powerful performance, its attention mechanism has high computational complexity, heavily relies on dedicated AI chips and large memory support, and is not friendly to deployment on computationally limited automotive embedded platforms, resulting in high practical application costs and difficulties. The second category is the mechanism based on view frustum projection and lift-splatter. Early methods of this type predicted the depth distribution of each pixel, lifted image features to a 3D view frustum space, and then projected them onto the BEV mesh through pooling operations, such as LSS and BEVDet series. Subsequent improvements introduced signals such as LiDAR point clouds to explicitly supervise depth prediction to improve geometric accuracy, such as BEVDepth. However, this method involves operations such as depth prediction, feature outer product, and voxel pooling for a large number of pixels, making the computational process relatively complex. It still requires high-performance parallel computing (such as CUDA) for acceleration, and the overall computational load remains significant. To pursue ultimate inference efficiency, a simplified implicit view transformation scheme has recently emerged. From M2 to FastBEV, these methods omit the complex depth prediction and voxel pooling steps, assuming that features are uniformly distributed along the camera ray and using camera geometry parameters to pre-calculate the mapping relationship between the image and the BEV space. While this method greatly improves the transformation speed, it inherently lacks geometric perception capabilities, especially in terms of the accuracy of estimating object distances and contours, because it completely abandons explicit modeling and utilization of scene depth information.

[0004] Existing technology discloses a behavior decision-making method and computer device for intelligent vehicles at disordered intersections. Based on Markov decision processes and combined with real-world traffic rules governing vehicles at disordered intersections, an action space A and a reward function R are designed. The intelligent vehicle acquires real-time LiDAR sensor information and bird's-eye view information of the disordered intersection from a simulation environment, constructing a state space S. A competitive dual-Q network, comprising a multilayer perceptron, convolutional neural network, and competitive neural network, is constructed. This network encodes and fuses the processed LiDAR sensor information and the bird's-eye view information of the disordered intersection (features extracted via spatial attention), decodes the input, and then inputs it into the competitive neural network to calculate the Q-value for decision-making. During the model evaluation phase, the intelligent vehicle makes decisions based on maximizing the Q-value, successfully navigating the disordered intersection. This method has a complex computational process and a large computational load. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies, namely high computational complexity and low perception accuracy, by providing a fast bird's-eye view model perception method based on decoupled depth supervision. This method features low computational complexity and high perception accuracy.

[0006] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows: A fast bird's-eye view model perception method based on decoupled depth supervision includes: S1: Obtain the training dataset; the training dataset includes multiple sets of training data; the training data includes multi-view images, LiDAR point clouds, and object features; S2: Select a set of training data from the training dataset as the first training data; S3: Input the multi-view images from the first training data into the feature extraction model to obtain image features and depth features; input the LiDAR point cloud from the first training data into the depth-assisted algorithm to obtain the depth ground truth map and edge weight map; S4: Input the image features and the depth features into the bird's-eye view detection model to obtain the predicted object features; S5: Calculate the total loss function value based on the object features in the first training data, the predicted object features, the depth ground truth map, the edge weight map, and the depth features; S6: Optimize the bird's-eye view detection model and the feature extraction model based on the total loss function value; S7: Select another set of training data from the training dataset as the new first training data, and repeat steps S3 to S6 until the total loss function value reaches the preset condition to obtain the trained feature extraction model and the trained bird's-eye view detection model. S8: Acquire the multi-view image to be perceived; S9: Preprocess the multi-view image to be perceived to obtain a preprocessed multi-view image; S10: Input the preprocessed multi-view image into the trained feature extraction model to obtain image features and depth features; S11: Input the image features and the depth features into the trained bird's-eye view detection model to obtain the features of the perceived object.

[0007] Furthermore, the feature extraction model includes: a 2D feature extraction unit and a depth prediction unit; The multi-view images in the first training data are input to the input end of the 2D feature extraction unit; the output end of the 2D feature extraction unit is connected to the input end of the depth prediction unit, the output end of the depth prediction unit outputs depth features; the output end of the 2D feature extraction unit outputs image features.

[0008] Furthermore, the 2D feature extraction unit includes: a feature backbone network and a pyramid network; The multi-view images in the first training data are input to the input end of the feature backbone network, the output end of the feature backbone network is connected to the input end of the pyramid network, and the output end of the pyramid network outputs image features; the output end of the pyramid network is connected to the input end of the depth prediction unit.

[0009] Furthermore, the depth prediction unit includes a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, and a third convolutional layer; The output of the 2D feature extraction unit is connected to the input of the first convolutional layer. The output of the first convolutional layer is connected to the input of the first activation layer. The output of the first activation layer is connected to the input of the second convolutional layer. The output of the second convolutional layer is connected to the input of the second activation layer. The output of the second activation layer is connected to the input of the third convolutional layer. The output of the third convolutional layer outputs depth features.

[0010] Furthermore, the bird's-eye view detection model includes: a coordinate system mapping unit, a bird's-eye view encoding unit, and a 3D detection head; The image features and the depth features are input to the input end of the coordinate system mapping unit; the output end of the coordinate system mapping unit is connected to the input end of the bird's-eye view encoding unit, the output end of the bird's-eye view encoding unit is connected to the input end of the 3D detection head, and the output end of the 3D detection head outputs the predicted object features.

[0011] Furthermore, depth-assisted algorithms include: S01: Generate depth ground truth from the lidar point cloud to obtain a depth ground truth map. ; S02: For the depth truth map Edge weights are calculated to obtain the edge weight map. .

[0012] Furthermore, the formula for calculating the total loss function value is as follows:

[0013] The CenterPoint detection loss represents the difference between the object features in the first training data and the predicted object features. Indicates hyperparameters, This represents the fine-grained depth loss function.

[0014] Furthermore, the calculation formula for the fine-grained depth loss function is as follows:

[0015] n represents the number of valid pixels, and i represents the pixel index. The depth weight parameter represents pixel i. This represents the total number of categories in the predicted depth distribution interval, where c represents the category index. This represents the weight hyperparameter of category c. Indicates the focus parameter, This represents the predicted depth of pixel i in category c. Indicates the depth of pixel i in category c;

[0016] Represents the edge weight graph;

[0017] D represents the depth truth map;

[0018] Represents depth features.

[0019] A fast bird's-eye view model perception system based on decoupled depth supervision, comprising: Dataset acquisition module: Acquires the training dataset; the training dataset includes multiple sets of training data; the training data includes multi-view images, LiDAR point clouds, and object features; Training data selection module: Selects a set of training data from the training dataset as the first training data; Feature extraction module: Input the multi-view images from the first training data into the feature extraction model to obtain image features and depth features; input the LiDAR point cloud from the first training data into the depth-assisted algorithm to obtain the depth ground truth map and edge weight map; Feature detection module: Inputs the image features and depth features into the bird's-eye view detection model to obtain predicted object features; Loss function calculation module: Calculates the total loss function value based on the object features in the first training data, the predicted object features, the depth ground truth map, the edge weight map, and the depth features; Optimization module: Optimizes the bird's-eye view detection model and the feature extraction model based on the total loss function value; Repeated training module: Select another set of training data from the training dataset as the new first training data, and repeat the process until the total loss function value reaches the preset condition, so as to obtain the trained feature extraction model and the trained bird's-eye view detection model. Image acquisition module: Acquires multi-view images to be perceived; Preprocessing module: preprocesses the multi-view images to be perceived to obtain preprocessed multi-view images; Feature extraction and inference module: Input the preprocessed multi-view images into the trained feature extraction model to obtain image features and depth features; Feature detection and inference module: Input the image features and the depth features into the trained bird's-eye view detection model to obtain the features of the perceived object.

[0020] A computer-readable storage medium includes a program for a fast bird's-eye view model perception method based on decoupled depth supervision. When executed by a processor, the program implements the steps of the fast bird's-eye view model perception method based on decoupled depth supervision.

[0021] Compared with the prior art, the beneficial effects of the present invention are: This invention introduces a deep-assisted algorithm during the training phase, thereby significantly improving the model's spatial geometry awareness during inference. Simultaneously, the deep-assisted algorithm does not participate in computation during inference, enabling accuracy improvements without sacrificing the efficiency of the original model. Attached Figure Description

[0022] Figure 1 The flowchart is provided for Example 1, which describes a fast bird's-eye view model perception method based on decoupled depth supervision.

[0023] Figure 2 This is a schematic diagram of the training process for a fast bird's-eye view model perception method based on decoupled depth supervision, as provided in Example 1.

[0024] Figure 3 This is a schematic diagram of the inference process of a fast bird's-eye view model perception method based on decoupled depth supervision, as provided in Example 1.

[0025] Figure 4 A flowchart of the depth-assisted algorithm provided in Example 1.

[0026] Figure 5 This is a schematic diagram of the depth truth map provided in Example 1.

[0027] Figure 6 A schematic diagram of the edge weight map provided in Example 1.

[0028] Figure 7 This is a schematic diagram of the perceived image provided in Example 1.

[0029] Figure 8 This is a schematic diagram illustrating the features of a sensed object provided in Example 1. Detailed Implementation

[0030] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0031] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0032] Example 1: like Figure 1 , Figure 2 , Figure 3 As shown, a fast bird's-eye view model perception method based on decoupled depth supervision includes: S1: Obtain the training dataset; the training dataset includes multiple sets of training data; the training data includes multi-view images, LiDAR point clouds, and object features; S2: Select a set of training data from the training dataset as the first training data; S3: Input the multi-view images from the first training data into the feature extraction model to obtain image features and depth features; input the LiDAR point cloud from the first training data into the depth-assisted algorithm to obtain the depth ground truth map and edge weight map; S4: Input the image features and the depth features into the bird's-eye view detection model to obtain the predicted object features; S5: Calculate the total loss function value based on the object features in the first training data, the predicted object features, the depth ground truth map, the edge weight map, and the depth features; S6: Optimize the bird's-eye view detection model and the feature extraction model based on the total loss function value; S7: Select another set of training data from the training dataset as the new first training data, and repeat steps S3 to S6 until the total loss function value reaches the preset condition to obtain the trained feature extraction model and the trained bird's-eye view detection model. S8: Acquire the multi-view image to be perceived; S9: Preprocess the multi-view image to be perceived to obtain a preprocessed multi-view image; S10: Input the preprocessed multi-view image into the trained feature extraction model to obtain image features and depth features; S11: Input the image features and the depth features into the trained bird's-eye view detection model to obtain the features of the perceived object.

[0033] Furthermore, the feature extraction model includes: a 2D feature extraction unit and a depth prediction unit; The multi-view images in the first training data are input to the input end of the 2D feature extraction unit; the output end of the 2D feature extraction unit is connected to the input end of the depth prediction unit, the output end of the depth prediction unit outputs depth features; the output end of the 2D feature extraction unit outputs image features.

[0034] Furthermore, the 2D feature extraction unit includes: a feature backbone network and a pyramid network; The multi-view images in the first training data are input to the input end of the feature backbone network, the output end of the feature backbone network is connected to the input end of the pyramid network, and the output end of the pyramid network outputs image features; the output end of the pyramid network is connected to the input end of the depth prediction unit.

[0035] In one specific embodiment, a ResNet-50 network is used as the backbone network, combined with a Feature Pyramid Network (FPN) as the neck network to form an image feature extractor. For each viewpoint image... Multi-scale feature maps are extracted and fused with the features output by the FPN to obtain the final feature map. ∈ ,in .

[0036] It should be noted that image features integrate low-level detailed information with high-level semantic information.

[0037] Furthermore, the depth prediction unit includes a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, and a third convolutional layer; The output of the 2D feature extraction unit is connected to the input of the first convolutional layer. The output of the first convolutional layer is connected to the input of the first activation layer. The output of the first activation layer is connected to the input of the second convolutional layer. The output of the second convolutional layer is connected to the input of the second activation layer. The output of the second activation layer is connected to the input of the third convolutional layer. The output of the third convolutional layer outputs depth features.

[0038] In one specific embodiment, the first and second convolutional layers are 3x3 convolutional layers, the first and second activation layers are ReLU activation layers, and the third convolutional layer is a 1x1 convolutional layer.

[0039] It should be noted that the depth prediction unit predicts a discrete depth distribution probability for each feature point in the image feature map. This means determining the probability that a pixel belongs to a different depth range.

[0040] Furthermore, the bird's-eye view detection model includes: a coordinate system mapping unit, a bird's-eye view encoding unit, and a 3D detection head; The image features and the depth features are input to the input end of the coordinate system mapping unit; the output end of the coordinate system mapping unit is connected to the input end of the bird's-eye view encoding unit, the output end of the bird's-eye view encoding unit is connected to the input end of the 3D detection head, and the output end of the 3D detection head outputs the predicted object features.

[0041] In one specific embodiment, an offline pre-computed lookup table (LUT) is used to map the image coordinate system to the BEV coordinate system. Based on the camera geometry parameters, the corresponding region for each BEV grid position (x, y) in each camera image is calculated. Using the pre-computed LUT, the 2D feature maps of each viewpoint are... The features are quickly aggregated into the BEV space, and the features of the first projection are directly used in the intersection areas of each view to form the initial BEV features. ∈ , where X*Y is the BEV grid resolution.

[0042] The bird's-eye view encoding unit uses a BEV encoder consisting of four contiguous residual convolutional blocks to process the initial BEV features. Further encoding yields enhanced BEV features. .

[0043] The 3D detection head is based on the CenterPoint detection framework, which includes classification and regression branches: the classification branch outputs a heatmap of the object's center. The regression branch outputs the object's position, size, orientation angle, velocity, and other attributes.

[0044] It should be noted that in the coordinate system mapping unit, image features are quickly converted into features in the bird's-eye view (BEV) space through an efficient view transformation module based on a pre-computed lookup table (LUT). This module utilizes camera geometric priors, avoiding complex real-time depth prediction and voxel pooling operations, resulting in extremely high efficiency. The data is then enhanced by a BEV encoder and input into a task head (such as a 3D detection head) to obtain the prediction result for the main task (such as a 3D bounding box). Furthermore, such as Figure 4 As shown, the depth-assisted algorithm includes: S01: Generate depth ground truth values ​​for the lidar point cloud to obtain, as shown below. Figure 5 The depth truth map shown ; S02: For the depth truth map Perform edge weight calculation to obtain, as follows Figure 6 The edge weight map shown .

[0045] In one specific embodiment, the calculation method of the depth-assisted algorithm is as follows: By using LiDAR point cloud data and back-projecting it onto the image plane using camera intrinsic and extrinsic parameters, a sparse depth ground truth map is generated. .

[0046] Extract edge information from images at various viewpoints and calculate their gradient magnitude maps G (e.g., the image is processed through a Sobel operator layer or a small edge detection CNN; or directly from the depth ground truth map generated from the point cloud). The gradients Gx and Gy are calculated, and then the gradient magnitude map is calculated. An edge weight map W is generated through a nonlinear transformation (such as the exponential function = exp(G / σ)). This map gives pixels at object edges and in depth discontinuous regions a much higher weight than pixels in flat regions.

[0047] Edge weight map calculation (this example uses gradient calculation on the depth ground truth map generated from the point cloud to obtain edge weights; alternatively, images can be obtained using the Sobel operator): For depth ground truth map Perform downsampling to Resolution.

[0048] Calculating gradient magnitude plots using the Sobel operator :

[0049]

[0050] Generate an edge weight map using an exponential function:

[0051] The hyperparameter σ is set to 1.0.

[0052] Furthermore, the formula for calculating the total loss function value is as follows:

[0053] The CenterPoint detection loss represents the difference between the object features in the first training data and the predicted object features. Indicates hyperparameters, This represents the fine-grained depth loss function.

[0054] Furthermore, the calculation formula for the fine-grained depth loss function is as follows:

[0055] n represents the number of valid pixels, and i represents the pixel index. The depth weight parameter represents pixel i. This represents the total number of categories in the predicted depth distribution interval, where c represents the category index. This represents the weight hyperparameter of category c. Indicates the focus parameter, This represents the predicted depth of pixel i in category c. Indicates the depth of pixel i in category c;

[0056] Represents the edge weight graph;

[0057] D represents the depth truth map;

[0058] Represents depth features.

[0059] In one specific embodiment, λ is set to 0.5. The AdamW optimizer is used with an initial learning rate of 2e-4, a cosine annealing strategy, a batch size of 8, and 20 training epochs.

[0060] In one specific embodiment, the preprocessing process is as follows: acquiring multi-view image sequences from the vehicle-mounted surround-view camera system. Where V is the number of cameras (usually 6), and the image resolution is H×W×3.

[0061] Using synchronously acquired lidar point cloud data, through camera intrinsic and extrinsic parameter matrices , The 3D point cloud is projected onto the image planes of each camera to generate a sparse depth ground truth map. :

[0062] Where (u,v) are the image coordinates, and (X,Y,Z) are the point cloud coordinates in the world coordinate system.

[0063] A fast bird's-eye view model perception system based on decoupled depth supervision, comprising: Dataset acquisition module: Acquires the training dataset; the training dataset includes multiple sets of training data; the training data includes multi-view images, LiDAR point clouds, and object features; Training data selection module: Selects a set of training data from the training dataset as the first training data; Feature extraction module: Input the multi-view images from the first training data into the feature extraction model to obtain image features and depth features; input the LiDAR point cloud from the first training data into the depth-assisted algorithm to obtain the depth ground truth map and edge weight map; Feature detection module: Inputs the image features and depth features into the bird's-eye view detection model to obtain predicted object features; Loss function calculation module: Calculates the total loss function value based on the object features in the first training data, the predicted object features, the depth ground truth map, the edge weight map, and the depth features; Optimization module: Optimizes the bird's-eye view detection model and the feature extraction model based on the total loss function value; Repeated training module: Select another set of training data from the training dataset as the new first training data, and repeat the process until the total loss function value reaches the preset condition, so as to obtain the trained feature extraction model and the trained bird's-eye view detection model. like Figure 7 As shown, the image acquisition module acquires multi-view images to be perceived; Preprocessing module: preprocesses the multi-view images to be perceived to obtain preprocessed multi-view images; Feature extraction and inference module: Input the preprocessed multi-view images into the trained feature extraction model to obtain image features and depth features; like Figure 8 As shown, the feature detection and inference module inputs the image features and the depth features into the trained bird's-eye view detection model to obtain the features of the perceived object.

[0064] A computer-readable storage medium includes a program for a fast bird's-eye view model perception method based on decoupled depth supervision. When executed by a processor, the program implements the steps of the fast bird's-eye view model perception method based on decoupled depth supervision.

[0065] It should be noted that this method significantly enhances the model's understanding of scene spatial structure by introducing targeted geometric supervision during the training phase, especially in challenging scenarios such as object edges and small, distant targets, resulting in a substantial improvement in detection accuracy. By placing the accuracy optimization entirely within the training phase, requiring no modifications to the inference phase, performance improvements are achieved without increasing resource consumption, making it extremely friendly to embedded deployments. This optimization framework, as a plug-and-play solution, can be widely applied to various BEV perception models based on implicit view transformations, without relying on any specific network architecture.

[0066] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A fast bird's-eye view model perception method based on decoupled depth supervision, characterized in that, include: S1: Obtain the training dataset; the training dataset includes multiple sets of training data; The training data includes multi-view images, LiDAR point clouds, and object features; S2: Select a set of training data from the training dataset as the first training data; S3: Input the multi-view images from the first training data into the feature extraction model to obtain image features and depth features; The lidar point cloud from the first training data is input into the depth-assisted algorithm to obtain the depth ground truth map and the edge weight map; S4: Input the image features and the depth features into the bird's-eye view detection model to obtain the predicted object features; S5: Calculate the total loss function value based on the object features in the first training data, the predicted object features, the depth ground truth map, the edge weight map, and the depth features; S6: Optimize the bird's-eye view detection model and the feature extraction model based on the total loss function value; S7: Select another set of training data from the training dataset as the new first training data, and repeat steps S3 to S6 until the total loss function value reaches the preset condition to obtain the trained feature extraction model and the trained bird's-eye view detection model. S8: Acquire the multi-view image to be perceived; S9: Preprocess the multi-view image to be perceived to obtain a preprocessed multi-view image; S10: Input the preprocessed multi-view image into the trained feature extraction model to obtain image features and depth features; S11: Input the image features and the depth features into the trained bird's-eye view detection model to obtain the features of the perceived object.

2. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 1, characterized in that, The feature extraction model includes: a 2D feature extraction unit and a depth prediction unit; The multi-view images in the first training data are input to the input end of the 2D feature extraction unit; the output end of the 2D feature extraction unit is connected to the input end of the depth prediction unit, the output end of the depth prediction unit outputs depth features; the output end of the 2D feature extraction unit outputs image features.

3. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 2, characterized in that, The 2D feature extraction unit includes: a feature backbone network and a pyramid network; The multi-view images in the first training data are input to the input end of the feature backbone network, the output end of the feature backbone network is connected to the input end of the pyramid network, and the output end of the pyramid network outputs image features; the output end of the pyramid network is connected to the input end of the depth prediction unit.

4. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 2, characterized in that, The depth prediction unit includes a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, and a third convolutional layer; The output of the 2D feature extraction unit is connected to the input of the first convolutional layer. The output of the first convolutional layer is connected to the input of the first activation layer. The output of the first activation layer is connected to the input of the second convolutional layer. The output of the second convolutional layer is connected to the input of the second activation layer. The output of the second activation layer is connected to the input of the third convolutional layer. The output of the third convolutional layer outputs depth features.

5. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 1, characterized in that, The bird's-eye view detection model includes: a coordinate system mapping unit, a bird's-eye view encoding unit, and a 3D detection head; The image features and the depth features are input to the input end of the coordinate system mapping unit; the output end of the coordinate system mapping unit is connected to the input end of the bird's-eye view encoding unit, the output end of the bird's-eye view encoding unit is connected to the input end of the 3D detection head, and the output end of the 3D detection head outputs the predicted object features.

6. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 1, characterized in that, Deep-assisted algorithms include: S01: Generate depth ground truth from the lidar point cloud to obtain a depth ground truth map. ; S02: For the depth truth map Edge weights are calculated to obtain the edge weight map. .

7. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 6, characterized in that, The formula for calculating the total loss function value is as follows: The CenterPoint detection loss represents the difference between the object features in the first training data and the predicted object features. Indicates hyperparameters, This represents the fine-grained depth loss function.

8. The fast bird's-eye view model perception method based on decoupled depth supervision according to claim 7, characterized in that, The formula for calculating the fine-grained depth loss function is as follows: n represents the number of valid pixels, and i represents the pixel index. The depth weight parameter represents pixel i. This represents the total number of categories in the predicted depth distribution interval, where c represents the category index. This represents the weight hyperparameter of category c. Indicates the focus parameter, This represents the predicted depth of pixel i in category c. Indicates the depth of pixel i in category c; Represents the edge weight graph; D represents the depth truth map; Represents depth features.

9. A fast bird's-eye view model perception system based on decoupled depth supervision, applied to the bird's-eye view model perception method according to any one of claims 1 to 8, characterized in that, include: Dataset Acquisition Module: Acquires the training dataset; The training dataset includes multiple sets of training data; The training data includes multi-view images, LiDAR point clouds, and object features; Training data selection module: Selects a set of training data from the training dataset as the first training data; Feature extraction module: Inputs multi-view images from the first training data into the feature extraction model to obtain image features and depth features; The lidar point cloud from the first training data is input into the depth-assisted algorithm to obtain the depth ground truth map and the edge weight map; Feature detection module: Inputs the image features and depth features into the bird's-eye view detection model to obtain predicted object features; Loss function calculation module: Calculates the total loss function value based on the object features in the first training data, the predicted object features, the depth ground truth map, the edge weight map, and the depth features; Optimization module: Optimizes the bird's-eye view detection model and the feature extraction model based on the total loss function value; Repeated training module: Select another set of training data from the training dataset as the new first training data, and repeat the process until the total loss function value reaches the preset condition, so as to obtain the trained feature extraction model and the trained bird's-eye view detection model. Image acquisition module: Acquires multi-view images to be perceived; Preprocessing module: preprocesses the multi-view images to be perceived to obtain preprocessed multi-view images; Feature extraction and inference module: Input the preprocessed multi-view images into the trained feature extraction model to obtain image features and depth features; Feature detection and inference module: Input the image features and the depth features into the trained bird's-eye view detection model to obtain the features of the perceived object.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a program for a fast bird's-eye view model perception method based on decoupled depth supervision. When the fast bird's-eye view model perception method based on decoupled depth supervision is executed by a processor, it implements the steps of the fast bird's-eye view model perception method based on decoupled depth supervision as described in any one of claims 1 to 8.

Images