A polar coordinate-based 3D-BEV lane line perception method

Abstract

The application relates to the technical field of lane line detection, in particular to a 3D-BEV lane line perception method based on polar coordinates, which comprises the following steps: 3D lane line detection is carried out by using polar coordinates to replace the traditional method based on the rectangular coordinate system, and a sliding long strip window convolution is used to adapt to the method of the polar coordinate system, the application can accurately identify the lane line in the picture in the polar coordinate system and accurately position in the 3D space, the false detection rate is greatly reduced, the radial distribution characteristics of the perspective projection can be more naturally matched, and the modeling precision of the lane line geometric structure is significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of lane line detection, and particularly relates to a 3D-BEV lane line perception method based on polar coordinates. BACKGROUND

[0002] Lane line detection technology aims to accurately extract the lane contour on the road and provide key drivable area information for autonomous driving systems, which is the core link to realize autonomous driving. In recent years, lane line detection technology has made unprecedented progress under the promotion of deep neural networks. To meet different needs of autonomous driving scenarios, existing researches are mainly divided into 2D-based lane line detection and 3D-based lane line detection. The 2D-based lane line detection usually focuses on real-time detection, and constructs a lightweight network to output planar lane lines on 2D images. This method has high inference speed and is suitable for direct deployment on vehicle platforms. However, the 2D detection method has obvious limitations: since it only relies on the planar information of a single image, the output lane lines lack depth and spatial structure information, and it is difficult to accurately reflect the three-dimensional geometric relationship of the real road. For example, in the scenarios of ramps, curves or complex intersections, 2D detection may not be able to distinguish the real topological structure of the near and far lanes, resulting in lane line projection distortion or misjudgment. Such planar output is difficult to meet the needs of autonomous driving systems for accurate environmental perception, especially in scenarios that require prediction of vehicle motion trajectory or high-precision path planning. In contrast, 3D-based lane line detection usually focuses more on obtaining more effective information from the picture to improve the quality of lane line detection. This method can reconstruct the three-dimensional geometric structure of the lane, thus more accurately reflecting the topological relationship of the real road. 3D detection can not only provide the planar position of the lane line, but also output its height, curvature, slope and other key three-dimensional attributes, which is crucial for autonomous driving systems to understand the actual spatial layout of the road. For example, in the case of road surface inclination or slope, the car can understand the road situation in time and control the active suspension to keep the passengers in the highest comfort. Although the computational complexity is high, 3D detection is more in line with the core needs of autonomous driving for high-precision and robust environmental modeling.

[0003] Current 3D-based researches mainly focus on capturing global information and modeling more accurate geometric consistency, focusing on eliminating perspective effects and other issues. For example, 3D-LaneNet and Gen-LaneNet use inverse perspective mapping (IPM) to project image feature maps to the ground plane to generate BEV feature maps, and convert 2D lanes into 3D lanes, but the IPM-based method is heavily dependent on accurate road plane assumptions, and is prone to projection errors in complex terrain or slope change scenarios; ONCE-3DLanes captures the geometric shape of the road line by using a multi-stage feature extraction network, and estimates the distance between the vehicle and the road by using monocular vision technology, but the multi-stage feature extraction network increases the computational complexity, which is difficult to meet the real-time requirements. In addition, the modeling of the topological relationship of the lane line in the existing methods is mostly insufficient, and the performance is poor in complex scenes such as intersections and merging areas; many methods lack effective depth estimation optimization mechanisms, resulting in insufficient vertical direction positioning accuracy. These deficiencies restrict the practical application effect of 3D lane detection technology in the automatic driving system.

[0004] In addition to the above limitations, the traditional 3D-based lane line detection method usually needs to preset a rectangular coordinate 3D space when predicting, so as to predict the lane line discrete points in the space. This way is not suitable for the information capture mode of ordinary cameras. Specifically, the camera maps the environment light on the pixel points in the perspective view. When the pixel information is inversely perspective transformed, the image information will show obvious angle range distribution. In addition, the external information obtained by the camera has a sparse characteristic, that is, the near information occupies more pixel points, and the far information occupies fewer pixel points. This radial distribution characteristic from the camera origin conflicts with the prediction in the rectangular coordinate system in the traditional method. These deficiencies jointly restrict the practical application effect and popularization of the current 3D lane line detection technology in the automatic driving system. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a 3D-BEV lane line perception method based on polar coordinates to solve the mismatching problem between the traditional rectangular coordinate system and the camera imaging characteristics.

[0006] In order to achieve the above purpose, the present application provides a 3D-BEV lane line perception method based on polar coordinates, comprising the following steps:

[0007] S1, acquiring an image, inputting the image into a ResNet-50 backbone network to extract primary features, and receiving the primary features through a feature pyramid network and further extracting multi-level features;

[0008] S2, input the multi-level feature into the polar coordinate encoder, calculate the field of view angle through the camera parameter, and calculate the polar coordinate angle of the pixel column based on the field of view angle;

[0009] S3, polar coordinate encoding is performed through the polar coordinate angle and the field of view angle, and a projection function is used to convert it into a high-dimensional position encoding;

[0010] S4, the multi-level feature is added to the high-dimensional position encoding, and a sliding window is used for angle-by-angle convolution to generate a polar coordinate feature;

[0011] S5, the multi-level feature is obtained again, and a multi-layer convolutional neural network is used to further extract lane line features, predict instance activation maps and perform weighted aggregation to generate instance features;

[0012] S6, the polar coordinate feature is globally pooled, spliced with the instance feature, and linearly projected to generate a lane line query, and a self-attention mechanism is used to explicitly model the global topological structure of the lane line;

[0013] S7, a set of binary masks is initialized, a spatial position encoding is generated through a sinusoidal position encoding function, and a projection is performed using an adaptive position encoding mapping layer;

[0014] S8, a 3D reference point is generated, a deformable attention mechanism is used for iterative optimization, and a three-dimensional coordinate prediction is corrected to obtain a detection result.

[0015] Preferably, step S2 specifically comprises:

[0016] First, a set of neural networks are used to further extract features, so that the size of the multi-level feature map is reduced to a reasonable range to obtain a feature map F';

[0017] For each spatial position on the feature map F', a normalized Cartesian coordinate system is established, and a position mapping relationship between the Cartesian coordinate and the polar coordinate is established;

[0018] The radial distance of perspective perception and the geometric prior of lane line direction rotation are explicitly injected into the feature map F', and the feature map F' is converted to polar coordinates;

[0019] The internal parameters of the camera are read, and the field of view angle of the image is calculated using the focal length of the x-axis;

[0020] For each position in the feature map, the polar coordinate angle of each column thereof is calculated according to the field of view angle.

[0021] Preferably, step S3 specifically comprises:

[0022] The distance between each position point and the center point of the feature map is calculated using the perspective perception prior, and the polar coordinate encoding parameters are obtained by combining the polar coordinate angle of each column and the distance between each position point and the center point of the feature map.

[0023] The polar coordinate space position encoding parameters are converted into high-dimensional features using a learnable projection function.

[0024] Preferably, step S5 specifically comprises:

[0025] The instance extractor processes the input features through a multi-layer convolutional neural network and predicts an instance activation map for a given multi-level feature map, and the spatial features are weighted and aggregated through the normalized instance activation map to obtain instance-level features.

[0026] Preferably, step S6 specifically comprises:

[0027] The mask branch generates spatial features through a convolutional neural network, the convolution kernel weight predicted by the instance branch is multiplied with the mask features to generate a final instance mask, and the instance features are further used for classification and confidence scoring, and an additional loss function is used to supervise the parameters in the instance activation, the classification loss is calculated using the focal loss, the mask loss is calculated using the overlap loss, and the instance existence loss is calculated using the binary cross-entropy loss, and then the losses are weighted using a weighting factor.

[0028] Preferably, step S7 specifically comprises:

[0029] The polar coordinate decoder first generates position encoding of a deformable attention mechanism, initializes a binary mask with all 0s according to the shape for a given input image feature, and then generates spatial position encoding through a sinusoidal position encoding function:

[0030] Then, the input instance features and polar coordinate features are spliced to generate an initial query in a linear projection manner.

[0031] Preferably, step S8 specifically comprises:

[0032] The reference point coordinates are generated, and the training process is divided into two stages: a generation stage and an iteration stage. In the generation stage, the normalized coordinates of the XZ plane are predicted through the query vector:

[0033] In the iteration stage, each layer of the polar coordinate decoder predicts the offset through the regression branch, and then realizes stable updating through inverse Sigmoid transformation:

[0034] Multi-scale deformable attention is performed, and in the polar coordinate decoder layer, the attention weight and the sampling offset are predicted through the query vector:

[0035] In the multi-scale deformable attention mechanism, each layer of the decoder predicts the coordinate offset through the regression branch, and finally updates the projection matrix from 3D to 2D by predicting the pitch angle and the height, and the projection matrix is used for position encoding calculation of the next layer;

[0036] In the detection, the detection head is used for the query of the last layer polar coordinate decoder output to generate a final prediction result.

[0037] Preferably, the method further comprises the following steps:

[0038] If in the model training stage, after the instance-level features are extracted, an independent detection head is used to predict the mask and classification information, and the extraction of the instance-level features is supervised by comparing with the true value of the data set.

[0039] Advantages of the present application:

[0040] 1. The present application breaks through the technical bottleneck that the traditional Cartesian coordinate system does not match the imaging characteristics of the camera by modeling the 3D lane line detection space as a polar coordinate system (radial distance, azimuth angle, pitch angle) with the camera optical center as the origin, so that the detection framework is more naturally matched with the radial distribution characteristics of the perspective projection, and the modeling accuracy and detection stability of the lane line geometric structure are significantly improved.

[0041] 2. The present application innovatively uses a sliding long strip window convolution instead of a traditional global convolution, and effectively solves the truncation problem of long strip lane line features in a local window by aligning the radial distribution characteristics of the lane line in the axial direction.

[0042] 3. The present application constructs a bidirectional enhancement architecture by injecting perspective-aware geometric priors through the polar coordinate encoder and combining the global topological modeling capability of the instance extractor, which solves the technical problem of feature ambiguity in the multi-lane instance confusion and occlusion scene, and greatly improves the lane line recognition accuracy in complex road scenes. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only illustrate the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0044] Fig. 1 The figure is a schematic diagram of the overall flow of the embodiment of the present application.

[0045] Fig. 2 The figure is a schematic diagram of the polar coordinate encoder of the embodiment of the present application.

[0046] Fig. 3 The figure is a schematic diagram of the instance extractor of the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will further illustrate the present application in detail by combining specific embodiments.

[0048] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the present application shall have the common meaning understood by one of ordinary skill in the art to which the present application pertains. The terms "first", "second", and similar terms used in the present application do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are only used to indicate relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships can also change accordingly.

[0049] As shown in Figs. 1-3 The polar coordinate-based 3D-BEV lane line perception method provided by the embodiments of the present application comprises the following steps:

[0050] S1, an image preprocessing module acquires image data, extracts primary features using a ResNet-50 network, and receives stored features and further extracts multi-level features using a feature pyramid network;

[0051] S2, a polar coordinate encoder acquires the multi-level features, calculates the field of view angle through camera parameters, and calculates the polar coordinate angle of the pixel column based on the field of view angle;

[0052] S3, polar coordinate encoding is performed through the polar coordinate angle and the field of view angle, and a projection function is used to convert it into high-dimensional position encoding;

[0053] S4, the multi-level features and the high-dimensional position encoding are added, and angle-by-angle convolution is performed using a sliding window to generate polar coordinate features;

[0054] S5, an instance extractor acquires the multi-level features, further extracts lane line features using a multi-layer convolutional neural network, predicts instance activation maps and performs weighted aggregation to generate instance features;

[0055] S6, a polar coordinate decoder globally pools the polar coordinate features, splices them with the instance features, and generates lane line queries through linear projection, and uses a self-attention mechanism to explicitly model the global topological structure of the lane lines;

[0056] S7, a set of binary masks is initialized, spatial position encoding is generated through a sinusoidal position encoding function, and projection is performed using an adaptive position encoding mapping layer;

[0057] S8, generating 3D reference points, using deformable attention mechanism for iterative optimization, correcting three-dimensional coordinate prediction, obtaining detection results, a linear layer module linearly maps the information output from the polar coordinate decoder, and outputs lane line position and classification information.

[0058] Preferably, the polar coordinate encoder uses the normalized polar coordinate position encoding for any input feature map First, a set of neural networks is used to further extract features, so that the size of the feature map is reduced to a reasonable range, and the amount of calculation is reduced:

[0059] F' = Φ (F) (1)

[0060] In the formula, Φ represents a feature extraction network composed of 3 3 3 convolution layers.

[0061] Subsequently, for each spatial position (i, j) on the feature map, a normalized Cartesian coordinate system is established. This step is to establish the position mapping relationship between the Cartesian coordinate and the polar coordinate, so as to directly convert the Cartesian coordinate system to the polar coordinate system through the position encoding. The position encoding process of the Cartesian coordinate system is as follows:

[0062]

[0063] In the formula, (H', W') represents the size of the feature map F'.

[0064] After obtaining the position encoding of the feature map F', it needs to be converted to polar coordinates. At this time, since the depth information of the image feature is not directly extracted, it can only be converted to polar coordinate position encoding in 2D form. However, in the 2D plane, if only the previous sine and cosine encoding is mechanically transplanted to the position encoding in the polar coordinate, the generated embedding vector can be regarded as a nonlinear transformation of the Cartesian coordinate (x, y) in mathematical expression, rather than truly introducing new geometric priors. Therefore, a polar coordinate position encoding suitable for 2D images is proposed, which explicitly injects the geometric priors of the radial distance of perspective perception and the direction rotation of the lane line in the feature map. Specifically, based on the principle of perspective view, the lane appears to diverge from the vanishing point to the bottom of the image in the image, so for any feature pixel, use its distance from the center point of the feature map as another two-dimensional position encoding in addition to the polar angle, and use the numerical size to represent the distance, and use the positive and negative signs to represent the pixel area, which can effectively input the radial distance prior of perspective perception into the model.

[0065] As for the polar angle, the actual camera horizontal field of view angle is used as the range of polar angle position encoding. First, calculate the polar coordinate position encoding. For any image with size (H, W), read the camera internal parameter K as shown in the following formula:

[0066]

[0067] where f x and f y represent the focal length of x-axis and y-axis respectively, in pixel; c x and c y represent the principal point coordinates, which is the center of optical axis in image coordinate system.

[0068] Next, we use the x-axis focal length f x to calculate the field of view angle of the image:

[0069]

[0070] For each position (i, j) in the feature map, we calculate the polar coordinate angle of each column:

[0071]

[0072] And for the perspective-aware prior, we directly use the following formula:

[0073]

[0074] where r i,j represents the distance between each position point (i, j) in the feature map and the center point of the feature map, which represents the pixel point, and θ i,j represents the polar coordinate angle information of each position point in the feature map, which obtains the distance between each position point and the center point of the feature map and the polar coordinate angle of each column, that is, the polar coordinate encoding parameter.

[0075] Then we use the learnable projection function to convert the polar coordinate space position encoding parameter into high-dimensional features:

[0076] e r = Ψ r (r)

[0077] e θ = Ψ θ (θ) (7)

[0078] where Ψ r and Ψ θ represent the multi-layer perceptron of radius and angle respectively, and the final position encoding is realized by feature addition:

[0079]

[0080] Next, let the window size be ω and the step size be s, and the Polar Encoder performs sliding window convolution along the angle θ:

[0081]

[0082] where <·> denotes the circular index of the angular dimension, Δθ is the angular interval, and W is the learnable convolution kernel. The final output polar coordinate feature represents the geometric structure information of the image in polar coordinates.

[0083] Preferably, the instance extractor extracts the instance features for a given feature map by a multi-layer convolutional neural network G inst processes the input feature and predicts the instance activation map where N is the preset number of instances, and the process can be represented as:

[0084] M norm = σ(G inst (F)) (10)

[0085] where σ(·) is the sigmoid function, which ensures that the activation value is in the range of [0, 1].

[0086] by the normalized instance activation map M norm , the spatial feature is weighted and aggregated to obtain the instance-level feature

[0087] Q inst = M norm · F flat (11)

[0088] where is the flattened feature map.

[0089] Subsequently, the mask branch generates spatial features F mask by a convolutional neural network G where D is the dynamic convolution kernel dimension. The instance branch predicts the convolution kernel weight and matrix multiplies the mask feature to generate the final instance mask

[0090] P mask = W kernel · F mask (12)

[0091] At the same time, the instance feature f inst is further used for classification (K is the number of classes) and confidence score

[0092] P cls = MLP cls (f inst )

[0093] P obj = MLP obj (finst ) (13)

[0094] The parameters in instance activation are supervised using additional loss functions, respectively, the classification loss L cls , the mask loss L mask , the instance existence loss L obj , and then weighted by a weighting factor:

[0095] L = λ cls L cls + λ mask L mask + λ obj L obj (14)

[0096] Preferably, the polar coordinate decoder first needs to generate the position encoding of the deformable attention mechanism, for a given input image feature Initialize a binary mask Y ∈ {0, 1} with all zeros according to the shape of the input image B×H×W , and then generate the spatial position encoding through the sinusoidal position encoding function PE(·):

[0097] E pos = AdaptPos3D(PE(Y)) (15)

[0098] where AdaptPos3D(·) is an adaptive position encoding mapping layer containing convolution and activation function.

[0099] Then, the input instance-level feature and the polar coordinate feature are concatenated to generate the initial query in a linear projection manner:

[0100] Q = Concat(Q polar , Q inst )W q (16)

[0101] where W is a learnable weight.

[0102] Next, the reference point coordinates P ref ∈ [0, 1] B×N×K×3 are generated, and the training process is divided into two stages: the generation stage and the iteration stage. In the generation stage, the normalized coordinates of the XZ plane are predicted through the query vector:

[0103]

[0104] where σ(·) is the Sigmoid function, which ensures that the coordinates fall within the interval [0, 1].

[0105] In the iterative phase, each polar coordinate decoder predicts the offset through a regression branch:

[0106] ΔP pred = Linear(Q l ) (18)

[0107] Then, a stable update is achieved by using the inverse Sigmoid transformation:

[0108]

[0109] Next, multi-scale deformable attention is performed. In the polar coordinate decoder layer l, the attention weight and the sampling offset are predicted by querying the vector:

[0110]

[0111] In the multi-scale deformable attention mechanism, each layer of the decoder predicts the coordinate offset

[0112]

[0113] where σ(·) is the Sigmoid function, and σ -1 (·) is its inverse function.

[0114] Finally, the projection matrix from 3D to 2D is dynamically updated by predicting the pitch angle θ and the height z:

[0115]

[0116] The projection matrix is used for the position encoding calculation of the next layer.

[0117] During detection, the query output by the last layer of the polar coordinate decoder is used with the detection head to generate the final prediction result, and an MLP is used for 3D lane line position estimation:

[0118] [Δρ,Δr,Δz,v] = MLP reg (Q) (23)

[0119] where represents the position of the corresponding reference point in the last layer of the polar coordinate decoder; indicates whether the predicted point is valid in the image, and at this time, N point sets are obtained as 3D reference points. For the lane category, the maximum pooling method along the point dimension is used, followed by an MLP for each instance, as follows:

[0120] C = MLP cls(MaxPool(Q)) (24)

[0121] wherein, is the set of classification results, where K represents the number of classes. In the final prediction, lanes classified as "background" will be discarded. Here, the same 2D match results as in the instance extractor are used for 3D lane label assignment. This strategy ensures consistent 2D segmentation and 3D lane detection supervision for each query.

[0122] It should be understood by those of ordinary skill in the art that the above discussion of any of the embodiments is merely exemplary in nature and not intended to imply limitations on the scope of the application, including the claims; nor has the application been reduced to practice in any aspect unless otherwise indicated; nor does it preclude the presentation of different embodiments or the combination of technical features from different embodiments in the application; nor do the steps have to be implemented in any order; nor are there many other variations of the different aspects of the application as described above, which have not been provided in detail for the sake of brevity; nor are they intended to be limiting.

[0123] The present application is intended to cover all such alternatives, modifications, and variations as come within the scope of the broadest possible interpretation of the appended claims. Accordingly, any and all such modifications, variations, and equivalents that fall within the spirit and scope of the application are intended to be included within the scope of the application.

Claims

1. A 3D-BEV lane line perception method based on polar coordinates, characterized in that, Includes the following steps: S1. Acquire the image. Input the image into the ResNet-50 backbone network to extract primary features. Receive the primary features through the feature pyramid network and further extract multi-level features. S2. Input the multi-level features into the polar coordinate encoder, calculate the field of view using camera parameters, and calculate the polar coordinate angle of the pixel column based on the field of view. S3. Polar coordinate encoding is performed using polar coordinate angles and field of view angles, and then the projection function is used to convert it into high-dimensional position encoding. S4. Add the multi-level features to the high-dimensional positional encoding, and use a sliding window to perform angle-wise convolution to generate polar coordinate features; S5. Obtain multi-level features again, input the multi-level features into the instance extractor, the instance extractor processes the multi-level features by using a multi-level convolutional neural network and predicts the instance activation map, and performs weighted aggregation through the normalized instance activation map to generate instance features. S6. Globally pool the polar coordinate features, concatenate them with the instance features, and generate lane line queries through linear projection. Specifically, this includes: The polar coordinate decoder first generates positional codes for a deformable attention mechanism, given the input image features. Generate a binary mask initialized to all zeros based on the shape. Then, through the sinusoidal position encoding function Generate spatial location codes: ； In the formula An adaptive positional encoding mapping layer containing convolutions and activation functions; Then, input instance-level features. and polar coordinate features The data is then concatenated to generate the initial query using linear projection: ； In the formula These are learnable weights; S8. Generate 3D reference points, use deformable attention mechanism for iterative optimization, correct the 3D coordinate prediction, and obtain the detection results; Step S2 specifically includes: First, features are further extracted using a set of neural networks to reduce the size of the multi-level feature maps to a reasonable range, thus obtaining the feature maps. ; For feature maps For each spatial location, establish a normalized Cartesian coordinate system and establish the positional mapping relationship between Cartesian coordinates and polar coordinates; In feature map The geometric priors of radial distance and lane line direction rotation in the explicit injection of perspective perception are used to transform the feature map. Transform to polar coordinates; Read the camera's internal parameters and calculate the field of view of the image using the x-axis focal length; For each position in the feature map, calculate the polar coordinate angle of each column based on the field of view; Step S3 specifically includes: The distance between each location point and the center point of the feature map is calculated using perspective perception priors. The polar coordinate encoding parameters are obtained by combining the polar coordinate angle of each column and the distance between each location point and the center point of the feature map. The polar coordinate spatial location encoding parameters are converted into high-dimensional features using a learnable projection function.

2. The 3D-BEV lane line perception method based on polar coordinates according to claim 1, characterized in that, Step S5 specifically includes: The masking branch generates masking features through a convolutional neural network. The kernel weights predicted by the instance branch are multiplied with the masking features to generate the final instance mask. Meanwhile, the instance features are further used for classification and confidence scoring. Additional loss functions are used to supervise the parameters in instance activation. The classification loss is calculated using focus loss, the masking loss is calculated using overlap loss, and the instance existence loss is calculated using binary cross-entropy loss. Finally, a weighting factor is used to weight them.

3. The 3D-BEV lane line perception method based on polar coordinates according to claim 1, characterized in that, Step S8 specifically includes: The training process for generating reference point coordinates consists of two phases: a generation phase and an iteration phase. In the generation phase, the normalized coordinates of the XZ plane are predicted by querying the vector. During the iterative phase, each polar coordinate decoder predicts the offset through a regression branch and then uses an inverse sigmoid transform to achieve stable updates. Multi-scale deformable attention is performed in the polar coordinate decoder layer, where attention weights and sampling offsets are predicted via the query vector: In the multi-scale deformable attention mechanism, each layer of the decoder predicts the coordinate offset through a regression branch, and finally dynamically updates the 3D to 2D projection matrix by predicting the pitch angle and height. The projection matrix is ​​used for the position encoding calculation of the next layer. During detection, the query output by the last layer polar coordinate decoder is used with the detection head to generate the final prediction result.

4. The 3D-BEV lane line perception method based on polar coordinates according to claim 1, characterized in that, It also includes the following steps: If it is in the model training phase, after extracting instance-level features, an independent detection head, prediction mask, and classification information will be used to compare with the ground truth in the dataset to supervise the extraction and training of instance-level features.

Images