A lane line detection method for a multi-junction complex road scene

Abstract

The application discloses a lane line detection method for a multi-branch complex road scene, comprising the following steps: 1. obtaining a multi-branch scene data sample and performing pretreatment; 2. constructing a multi-branch complex road scene lane line detection network, comprising: an image feature extraction network based on an attention mechanism, a point cloud body column feature extraction network, a lane line generative adversarial network, a cross attention module, a perspective conversion module and a detection head; 3. extracting features from the data sample by using the image feature extraction network based on the attention mechanism and the point cloud body column feature extraction network; 4. generating fused features by combining the lane line generative adversarial network and the cross attention module; and 5. converting the fused features into BEV features by using the perspective conversion module and inputting the BEV features into the detection head to generate a lane line coordinate set. The application can fully extract multi-modal information, effectively solve the problems of long-distance dependence and depth jumping of lane lines, and thus improve the lane line detection effect in special scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving perception, and more particularly to a lane line detection method, especially applied to a complex road scene with multiple branch outlets. BACKGROUND

[0002] In the perception of road structure elements, lane line detection is an important part of environment perception for high-level automatic driving, which needs to perceive lane lines in various road environments. The autonomous driving vehicle selects the feasible area through the results of lane line detection, and provides a basis for downstream decision planning. However, the actual road environment is complex, and there are various types of lane lines. The changes in light conditions, object shadows, wear and tear, and lack of visual clues are all problems that lane line detection faces.

[0003] The lane line detection method is generally divided into two schemes: traditional method and deep learning. In the traditional method, the lane line detection method based on color features converts the commonly used RGB channel image to HSV color features, and then sets the threshold value manually to detect the lane line position; the lane line detection method combining edge detection and Hough transformation first converts the color image into a gray image, and then performs fuzzy processing, edge detection and Hough transformation. The traditional method can usually only detect the lane line in a simple scene, and may not be robust enough for complex situations. Compared with the traditional detection algorithm, the deep learning method is divided into segmentation-based scheme, anchor-based method and parameter-based method. The segmentation-based method models the lane line detection as a pixel-by-pixel classification problem, each pixel is classified as a lane line region or background, and image classification is realized at the pixel level, but in a multi-branch complex scene, due to the long-distance dependence and large depth change of the lane line, the model is prone to misjudgment; the anchor-based method usually uses a top-down method to predict the lane line, uses the prior knowledge that the lane line extends from the near to the far in the driving perspective, constructs a lane line instance, designs a line type Anchor, and regresses the offset of the sampling point and the pre-defined Anchor, and applies non-maximum suppression (NMS) to select the lane line with the highest confidence, but in a complex multi-branch scene, the extension trend of the lane line is not obvious, especially in the case of perspective change or complex road conditions, which leads to the loss of lane line information; the parameter-based method uses a pre-defined parameter curve, generally through polynomial curve regression, and outputs a polynomial representing each lane line, thereby detecting the lane line shape, but when the lane line shape is irregular (such as sharp turns, branches, etc.), the fitting result will be quite different from the actual situation, reducing the accuracy and practicability of the detection. SUMMARY

[0004] The present application is to avoid the deficiencies of the prior art, effectively solve the long-distance dependence and depth jump problem of lane lines in multi-branch complex road scenes, and provide a lane line detection method for multi-branch complex road scenes, so as to meet the requirements of lane line detection accuracy, recall rate and information integrity of multi-branch complex roads, and improve the lane line detection effect in special scenes.

[0005] In order to achieve the above-mentioned application purposes, the present application adopts the following technical solutions:

[0006] The lane line detection method for multi-branch complex road scenes has the following steps:

[0007] Step 1, obtaining data of multi-branch complex road scenes and preprocessing;

[0008] Step 1.1, constructing a lane line data set with lane line category labels in a multi-branch complex road scene , N is the total number of data samples, is the nth data sample, and the category label of the dth lane line in the nth data sample is denoted as , represents the nth complex road image, C, H and W represent the channel number, height and width of the complex road image respectively; represents the nth point cloud data, and , wherein, represents the true position information of the lane line in , represents the three-dimensional coordinates of the dth lane line in , represents the reflection intensity of the dth lane line, represents the maximum number of sampling points of the lane line;

[0009] Step 1.2, performing data enhancement processing on the nth complex road image to obtain the nth enhanced image ;

[0010] Step 1.3, performing data enhancement processing on the nth point cloud information to obtain the nth enhanced point cloud information ;

[0011] Step 2, constructing a multi-branch complex road scene lane line detection network, including: an image feature extraction network based on an attention mechanism, a point cloud body column feature extraction network, a lane line generative adversarial network, a cross-attention module, a view conversion module, and a detection head;

[0012] ​Step 2.1, the image feature extraction network pair based on the attention mechanism is used to process the n-th multi-scale lane line image feature

[0013] Step 2.2, the point cloud column feature extraction network pair is used to process the n-th lane line pseudo-image feature

[0014] Step 2.3, the lane line generation adversarial network is used to process the n-th lane line pseudo-image feature to obtain the n-th multi-scale lane line point cloud feature and the determination result the probability that the real point cloud feature is true ; the weighted loss for constructing the lane line generation adversarial network , so as to train the lane line generation adversarial network by using the gradient descent method, and calculate the weighted loss of the lane line generation adversarial network to update the network parameters until convergence, thereby obtaining the trained lane line generation adversarial network;

[0015] Step 2.4, the cross-attention module is used to process the n-th optimal multi-scale lane line point cloud feature output by the trained lane line generation adversarial network and the n-th multi-scale lane line image feature to obtain the n-th fusion feature

[0016] Step 2.5, the perspective conversion module is used to process to obtain the n-th lane line feature map , wherein is the number of channels of the lane line feature map, , is the height and width of the lane line feature map.

[0017] Step 2.6, the is input into the detection head, thereby generating the n-th data sample predicted lane line position set and the predicted lane line category , wherein represents the i-th three-dimensional coordinate point constituting the d-th predicted lane line in the n-th data sample in the BEV space, represents the maximum number of three-dimensional coordinate points of the predicted lane line, represents the d-th predicted lane line category.

[0018] ​​​​​The lane line detection method for the multi-junction complex road scene has the characteristics that the image feature extraction network of the attention mechanism in step 2.1 is composed of a parallel sub-convolutional layer, a global maximum pooling layer, a global average pooling, a feature splicing layer, and a coordinate attention mechanism layer.

[0019] In step 2.1.1, after being input into the feature extraction network based on the attention mechanism, the image is sequentially processed by a parallel sub-convolutional layer, a global maximum pooling layer, and a global average pooling layer, to obtain the nth global average feature map and the ith global maximum feature map .

[0020] In step 2.1.2, and are spliced by a feature splicing layer and are up-sampled to obtain the nth lane line splicing feature .

[0021] In step 2.1.3, the coordinate attention mechanism layer respectively pools the height and width directions of using formula (1.1) and formula (1.2) to obtain the nth height aggregation feature and the nth width aggregation feature .

[0022] (1.1)

[0023] (1.2)

[0024] In formula (1.1) and formula (1.2), represents the coordinate point feature in the hth row and the ith column in . represents the coordinate point feature in the jth row and the wth column in .

[0025] In step 2.1.4, after transposing and splicing with and performing dimension reduction processing, the result is input into the ReLu function for processing to obtain the nth lane line encoding feature .

[0026] In step 2.1.5, the coordinate attention mechanism layer processes ​​After segmentation along the height and width directions of the image, the width lane line encoding features are obtained and the height lane line encoding features , so as to obtain the attention weight along the width direction of the nth lane line detection by formula (1.3) and formula (1.4) respectively and the attention weight along the height direction .

[0027] (1.3)

[0028] (1.4)

[0029] In formula (1.3) and formula (1.4), , is a convolution operation along the height and width directions of the image respectively, is a Sigmoid function;

[0030] Step 2.1.6, the image features of the nth multi-scale lane line are obtained by formula (1.5) .

[0031] (1.5)

[0032] Further, the point cloud body column feature network in the step 2.2 is composed of a voxel division layer, an MLP layer, a convolution layer and a global maximum pooling layer;

[0033] Step 2.2.1, the voxel division layer divides the feature into P vertical columnar regions, and selects the features in the non-empty vertical columnar regions , wherein, represents the feature of the i-th non-empty vertical columnar region in , and M is the maximum number of non-empty vertical columnar regions;

[0034] Step 2.2.2, after sequentially passing through the MLP layer, the convolution layer and the global maximum pooling layer, the nth lane line pseudo image feature is output .

[0035] Further, the lane line generation adversarial network in the step 2.3 is composed of a generator G and a discriminator D, and the generator and the discriminator are both composed of a multi-layer convolution module, a deep convolution layer and a residual block;

[0036] Step 2.3.1, is input into the generator G, and after sequentially passing through the multi-layer convolution module, the deep convolution layer and the residual block, the nth generated layer feature is obtained, After the full connection layer and the normalization processing, the predicted n-th multi-scale point cloud feature is output , voxel information representing the predicted lane line position;

[0037] Step 2.3.2, and is input into the discriminator D and sequentially processed by a plurality of convolution modules, a deep convolution layer and a residual block to obtain an n-th discriminator layer feature , After the processing of the Sigmoid activation function, the probability that the final output is determined as a real point cloud feature ;

[0038] Step 2.3.3, the point cloud inversion loss of the generator G is constructed by formula (1.6) ;

[0039] (1.6)

[0040] In formula (1.6), represents the expected value of , represents the L1 loss, represents the loss weight parameter;

[0041] Step 2.3.4, the point cloud discrimination loss of the discriminator D is constructed by formula (1.7) ;

[0042] (1.7)

[0043] In formula (1.7), represents the expected value of ;

[0044] Step 2.3.5, the weighted loss of the lane line generative adversarial network is constructed by formula (1.8) ;

[0045] (1.8)

[0046] In formula (1.8), and are two weight parameters for balancing the influence of and .

[0047] Further, the step 2.4 includes:

[0048] Step 2.4.1, the multi-scale lane line image feature , optimal multi-scale point cloud feature Project into the same dimensional space, so as to obtain the query matrix by formula (1.9), formula (1.10), formula (1.11) , key matrix , value matrix :

[0049] (1.9)

[0050] (1.10)

[0051] (1.11)

[0052] In formula (1.9), formula (1.10), formula (1.11), , , are three projection matrices;

[0053] Step 2.4.2, generate attention weights between multi-modal lane line features by formula (1.12) :

[0054] (1.12)

[0055] In formula (1.12), denotes the dimension of the key matrix and the query matrix, denotes an activation function;

[0056] Step 2.4.3, after point multiplication of and , the n-th multi-modal intermediate feature is obtained.

[0057] Step 2.4.4, after splicing of and , the n-th fusion feature is obtained.

[0058] Further, the step 2.5 includes:

[0059] Step 2.5.1, set , wherein, , , are the step lengths of the x direction, the y direction and the z direction in the BEV space coordinate system, respectively, and the unit cells are constructed with the step lengths, so as to obtain the BEV space composed of a plurality of unit cells.

[0060] Step 2.5.2, the perspective conversion module converts the fusion feature After sequentially passing through a convolutional layer, normalization processing and sonftmax activation function processing, a depth probability distribution D of the nth lane line is obtained n , D n is obtained After splicing, the depth fusion feature of the nth is obtained ;

[0061] Step 2.5.3, D is projected into the BEV space to obtain the three-dimensional point cloud coordinates of the lane line, and the three-dimensional coordinate points of the lane line are distributed into the unit cell to obtain the lane line feature of the i th unit cell in the BEV space ;

[0062] After sequentially passing through a convolutional layer, a pooling layer, normalization, and ReLu function processing, a lane line feature map of the nth data sample is output .

[0063] The electronic device comprises a memory and a processor, and the feature lies in that the memory is used to store a program supporting the processor to execute the lane line detection method, and the processor is configured to execute the program stored in the memory.

[0064] The computer readable storage medium stores a computer program, and the feature lies in that the computer program is executed by the processor to execute the steps of the lane line detection method.

[0065] Compared with the prior art, the present application has the following advantages:

[0066] 1. The present application designs a multi-modal feature fusion module, which uses a cross attention module to fuse lane line point cloud features and lane line image features to obtain rich feature space and depth information of lane lines in a multi-branch scene, overcoming the problem that the prior art can only detect lane lines in a simple scene, thereby improving the robustness, adaptability and accuracy of lane line detection.

[0067] 2. The present application uses a coordinate attention module (CA) to extract image features and enhance lane line position weights, taking into account the long and thin characteristics of lane lines, overcoming the depth jump problem of lane lines in a multi-branch complex scene, and enhancing the effect of lane line feature extraction without increasing the computational amount of the detection network. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a complete lane line detection flowchart;

[0069] Figure 2 is a structure diagram of the coordinate attention mechanism;

[0070] Figure 3 A schematic diagram of a lane line detection result. DETAILED DESCRIPTION

[0071] In this embodiment, as shown in the figure, the flow of a lane line detection method for a multi-junction complex road scene includes the following steps: Figure 1

[0072] Step 1, obtaining data of a multi-junction complex road scene and preprocessing;

[0073] Step 1.1, under the multi-junction complex road scene, constructing a lane line data set with a category label of lane line N is the total number of data samples, is the nth data sample, the category label of the dth lane line in the nth data sample is denoted as , represents the nth complex road image, C, H and W represent the channel number, height and width of the complex road image respectively, in this example, C takes 3, and H and W both take 512; represents the nth point cloud data, and wherein, represents the real position information of the lane line in , represents the three-dimensional coordinates of the dth lane line in , represents the reflection intensity of the dth lane line, represents the maximum number of sampling points of the lane line.

[0074] Step 1.2, performing data enhancement processing on the nth complex road image , after scaling the input nth complex road image according to a factor of 0.8, adjusting back to the standard input size by using bilinear interpolation, then flipping at a ratio of 0.5; randomly occluding 20% of the total area by a rectangular mask, flipping the image order, finally performing Gaussian blur operation on the image data, obtaining the nth enhanced image ; in this example, the enhanced image and keep the same format.

[0075] Step 1.3, performing data enhancement processing on the nth point cloud information , randomly flipping the coordinate axis, randomly rotating the point cloud and randomly adding noise to the point cloud data to enhance the data, obtaining the nth enhanced point cloud information ​​​​.

[0076] Step 2, constructing a multi-junction complex road scene lane line detection network, comprising: an image feature extraction network based on an attention mechanism, a point cloud body feature extraction network, a lane line generative adversarial network, a cross-attention module, a perspective conversion module, and a detection head;

[0077] Step 2.1, the image feature extraction network based on the attention mechanism processes the image to obtain the image feature of the nth multi-scale lane line . ;

[0078] The image feature extraction network based on the attention mechanism in step 2.1 is composed of parallel sub-convolutional layers, global maximum pooling layers, global average pooling layers, feature concatenation layers, and a coordinate attention mechanism layer; in this embodiment, , the parallel convolutional layer output channel is 3, , and .

[0079] Step 2.1.1, inputting the image into the feature extraction network based on the attention mechanism, first sequentially passing through parallel sub-convolutional layers, then respectively inputting global maximum pooling layers and global average pooling layers, and correspondingly obtaining the nth global average feature map and the ith global maximum feature map .

[0080] Step 2.1.2, concatenating and together through feature concatenation layers and performing upsampling to obtain the nth lane line concatenation feature .

[0081] Step 2.1.3, the coordinate attention mechanism layer structure is as shown in Figure 2 , which respectively utilizes formula (1.1) and formula (1.2) to use (H, 1) and (1, W) pooling kernels to pool along the height and width directions, and correspondingly obtain the nth height aggregation feature and the nth width aggregation feature , overcoming the long distance dependence problem.

[0082] (1.1)

[0083] ​(1.2)

[0084] In formula (1.1) and formula (1.2), represents the coordinate point feature of the ith column of the hth row in the matrix; represents the coordinate point feature of the wth column of the jth row in the matrix.

[0085] Step 2.1.4, the is transposed and spliced with the after dimension reduction processing, input into the ReLu function for processing, and the nth lane line coding feature is obtained.

[0086] Step 2.1.5, the coordinate attention mechanism layer processes after being divided along the height and width directions of the image, the width lane line coding feature and the height lane line coding feature are obtained, so that the attention weight along the width direction of the nth lane line detection is obtained by formula (1.3) and the attention weight along the height direction is obtained by formula (1.4) .

[0087] (1.3)

[0088] (1.4)

[0089] In formula (1.3) and formula (1.4), , are convolution operations along the height and width directions of the image, respectively, and in this embodiment, a 1*1 convolution kernel is used for convolution, is a Sigmoid function.

[0090] Step 2.1.6, the image feature of the nth multi-scale lane line is obtained by formula (1.5) .

[0091] (1.5)

[0092] In this embodiment, the enhanced feature map with a size of 3*512*512 is input into the image feature extraction network based on the attention mechanism, first, the parallel convolution layer, global average pooling and global maximum pooling are used to process , and the global average feature map and the global maximum feature map with a size of 3*1*1 are obtained, respectively, then The splicing processing of the feature splicing layer and the upsampling operation are performed to obtain lane line splicing features with a size of 3x512x512 The (H, 1) and (1, W) pooling kernels are used to pool along the height and width directions to obtain 3x1x512 height aggregation features and 3x512x1 width aggregation features After transposition, splicing, and dimension reduction, the input is processed in the ReLu function to obtain lane line encoding features with a size of 3x1x1024 Finally, the lane line encoding features are processed by using a 1*1 convolution kernel for dimension increasing, and then combined with a sigmoid activation function to obtain attention weights in the width direction and attention weights in the height direction The obtained attention weights are multiplied with to obtain image features of multi-scale lane lines . .

[0093] Step 2.2, the point cloud body column feature extraction network processes to obtain the nth lane line pseudo-image feature ; wherein the point cloud body column feature network is composed of a voxel division layer, an MLP layer, a convolution layer, and a global maximum pooling layer; the point cloud body column feature network is composed of 1 voxel division layer, 2 MLP layers, 3 convolution layers, and 1 global maximum pooling layer; Step 2.2.1, the voxel division layer divides into P vertical columnar regions, and selects features in the non-empty vertical columnar regions

[0094] , wherein represents the features of the i-th non-empty vertical columnar region in, and M is the maximum number of non-empty vertical columnar regions; Step 2.2.2, after being sequentially processed by the MLP layer, the convolution layer, and the global maximum pooling layer, the nth lane line pseudo-image feature is output ; in this example, P is initialized as 150;

[0095] Step 2.3, the lane line generation adversarial network processes the nth lane line pseudo-image feature to obtain the nth multi-scale lane line point cloud feature and the determination result is the probability that the real point cloud feature is true

[0096] ​​​​​​The lane line generation adversarial network is composed of a generator G and a discriminator D, and both the generator and the discriminator are composed of R multi-layer convolution modules, I deep convolution layers and U residual blocks. In this example, R is 3, I is 2, and U is 6.

[0097] Step 2.3.1, The input is input into the generator G and sequentially processed by the multi-layer convolution module, the deep convolution layer and the residual block to obtain the nth generation layer feature , After the full connection layer and the normalization processing, the predicted nth multi-scale point cloud feature , represents the voxel information of the predicted lane line position.

[0098] Step 2.3.2, and The input is input into the discriminator D and sequentially processed by the multi-layer convolution module, the deep convolution layer and the residual block to obtain the nth discrimination layer feature , After the processing of the Sigmoid activation function, the final output is the probability of being judged as a real point cloud feature .

[0099] Step 2.3.3, the point cloud inversion loss of the generator G is constructed by formula (1.6) The point cloud inversion loss function is used to ensure that the generated point cloud position information is consistent with the real position.

[0100] (1.6)

[0101] In formula (1.6), represents the expected value of the predicted multi-scale lane line point cloud feature , represents the L1 loss, represents the weight hyperparameter of the loss.

[0102] Step 2.3.4, the point cloud discrimination loss of the discriminator D is constructed by formula (1.7) .

[0103] (1.7)

[0104] In formula (1.7), represents the expected value of .

[0105] Step 2.3.5, the weighted loss of the lane line generation adversarial network is constructed by formula (1.8) .

[0106] (1.8)

[0107] In formula (1.8), , are two weight parameters for balancing the influence of and ; in this embodiment, and are respectively 0.3 and 0.7.

[0108] Step 2.3.6, the lane line generation adversarial network is trained by using the gradient descent method, and the weighted loss of the lane line generation adversarial network is calculated to update the network parameters until converges, thereby obtaining the trained lane line generation adversarial network.

[0109] Step 2.4, the cross-attention module processes the n-th optimal multi-scale lane line point cloud feature output by the trained lane line generation adversarial network and the n-th multi-scale lane line image feature to obtain the n-th fusion feature .

[0110] Step 2.4.1, the multi-scale lane line image feature and the optimal multi-scale point cloud feature are projected into the same dimensional space, thereby obtaining the query matrix , the key matrix and the value matrix by using formula (1.9), formula (1.10) and formula (1.11):

[0111] (1.9)

[0112] (1.10)

[0113] (1.11)

[0114] In formula (1.9), formula (1.10) and formula (1.11), , , are three projection matrices.

[0115] Step 2.4.2, the attention weight between the multi-modal lane line features is generated by using formula (1.12):

[0116] (1.12) ​

[0117] In formula (1.12), denotes the dimension of the key matrix and the query matrix, denotes an activation function;

[0118] Step 2.4.3, the is multiplied by , and the nth multimodal intermediate feature is obtained.

[0119] Step 2.4.4, after the multimodal intermediate feature is spliced with , the nth fusion feature is obtained.

[0120] Step 2.5, the perspective conversion module processes , and the nth lane line feature map is obtained, wherein is the number of channels of the lane line feature map, , is the height and width of the lane line feature map;

[0121] Step 2.5.1, set , wherein , , are the step sizes of the x direction, the y direction and the z direction in the BEV space coordinate system, and the step size is 0.25 in the present example. A cell is constructed by the step size, so as to obtain a BEV space composed of 360x360x1 cells.

[0122] Step 2.5.2, the perspective conversion module processes the first 118 features of the fusion feature in turn through a convolution layer, normalization processing and a softmax activation function, and obtains the nth lane line depth probability distribution D n . n After D is spliced with , the nth depth fusion feature is obtained.

[0123] Step 2.5.3, project into the BEV space, and generate a representation of the pixels in the fusion feature whose lane line weight is greater than 0.85, so as to obtain a three-dimensional point cloud coordinate of the lane line, complete the conversion from the two-dimensional coordinate system to the three-dimensional coordinate system shared by all cameras, distribute the three-dimensional coordinate points of the lane line into the cells, and obtain the lane line feature of the ith cell in the BEV space. ;

[0124] Step 2.6, the incoming detection head, thereby generating the n th data sample under the multi-branch complex road scene predicted lane line position set and the predicted lane line category wherein, represents the i th three-dimensional coordinate point of the d th lane line in the n th data sample, i = 1, 2, 3, …, , represents the maximum number of three-dimensional coordinate points of the d th lane line, represents the category of the d th lane line, and is the classification label corresponding to the lane line position set , used to identify the type of the lane line, and in the embodiment, the lane line type is set to solid line, dashed line, dashed line-special (special line type such as variable guide lane line, speed reduction marking, etc., the center main line is dashed), solid line-special (special line type such as variable guide lane line, speed reduction marking, etc., the center main line is solid), and road edge.

[0125] In the embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0126] In the embodiment, a computer readable storage medium has a computer program stored thereon, and the computer program is run by a processor to execute the steps of the above method.

Claims

1. A lane line detection method for complex road scenarios with multiple intersections, characterized in that, Includes the following steps: Step 1: Acquire data for complex road scenarios with multiple intersections and perform preprocessing; Step 1.1: In complex road scenarios with multiple intersections, construct a lane line dataset with lane line category labels. N is the total number of data samples. For the nth data sample, the nth data sample The category label of the d-th lane line is denoted as , Let C represent the nth complex road image, where C, H, and W represent the number of channels, height, and width of the complex road image, respectively. Represents the nth point cloud data, and ,in, express The actual location information of the middle lane line , express The three-dimensional coordinates of the d-th lane line. This represents the reflection intensity of the d-th lane line. This indicates the maximum number of sampling points for the lane line; Step 1.2: For the nth complex road image Perform data augmentation to obtain the nth augmented image. ; Step 1.3: For the nth point cloud information Perform data augmentation to obtain the nth augmented point cloud information. ; Step 2: Construct a lane line detection network for complex road scenes with multiple intersections, including: an image feature extraction network based on an attention mechanism, a point cloud volumetric feature extraction network, a lane line generative adversarial network, a cross attention module, a viewpoint conversion module, and a detection head; Step 2.1: An image feature extraction network based on an attention mechanism... The image features of the nth multi-scale lane line are obtained through processing. ; Step 2.2: Point cloud volumetric feature extraction network Processing is performed to obtain the pseudo-image features of the nth lane line. ; Step 2.3: The lane line generative adversarial network processes the pseudo-image features of the nth lane line. The process is performed to obtain the nth multi-scale lane line point cloud feature. and the judgment result Probability of being a true point cloud feature Weighted loss used to construct lane line generative adversarial networks Therefore, the lane line generative adversarial network is trained using the gradient descent method, and the weighted loss of the lane line generative adversarial network is calculated. To update network parameters until The training continues until convergence, thus obtaining the trained lane line generative adversarial network. Step 2.4: The cross-attention module processes the nth optimal multi-scale lane line point cloud feature output by the trained lane line generative adversarial network. Image features of the nth multi-scale lane line Processing is performed to obtain the nth fusion feature. ; Step 2.5, Viewpoint switching module The process is performed to obtain the feature map of the nth lane line. ,in, It is the number of channels in the lane feature map. , It refers to the height and width of the lane line feature map; Step 2.6, The data is fed into the detection head to generate the nth data sample in a complex road scenario with multiple intersections. Predicted set of lane line positions With predicted lane line categories ,in, This represents the i-th three-dimensional coordinate point of the predicted d-th lane line in the n-th data sample within the BEV space. This represents the maximum number of three-dimensional coordinate points for the predicted lane lines. This indicates the predicted lane category d.

2. The lane line detection method for complex road scenarios with multiple intersections according to claim 1, characterized in that, The image feature extraction network for the attention mechanism in step 2.1 is composed of... Parallel subconvolutional layers, A global max pooling layer, Global average pooling, It consists of a feature splicing layer and a coordinate attention mechanism layer; Step 2.1.1, The input is processed sequentially in the attention-based feature extraction network. After processing by several parallel sub-convolutional layers, the data is then input separately. The global max pooling layer and After processing by each global average pooling layer, the nth global average feature map is obtained. and the i-th global maximum feature map ; Step 2.1.2 and Passing by together The feature stitching layers are stitched together and upsampled to obtain the nth lane line stitching feature. ; Step 2.1.3: The coordinate attention mechanism layer utilizes equations (1.1) and (1.2) respectively to... Pooling along the height and width directions yields the nth height-aggregated feature. Aggregate features with the nth width ; (1.1) (1.2) In equations (1.1) and (1.2), express The coordinate characteristics of the h-th row and i-th column; express The coordinate characteristics of the point in the j-th row and w-th column; Step 2.1.4, After transpose and After concatenation and dimensionality reduction, the data is input into a ReLU function for further processing to obtain the nth lane line encoding feature. ; Step 2.1.5, the coordinate attention mechanism layer... After segmenting the image along its height and width, the width lane line coding features are obtained. With height lane line coding features Thus, using equations (1.3) and (1.4), the attention weights for the nth lane detection along the width direction are obtained respectively. Attention weights along the height direction ; (1.3) (1.4) In equations (1.3) and (1.4), , These are convolution operations along the height and width directions of the image, respectively. For the Sigmoid function; Step 2.1.6: Use equation (1.5) to obtain the image features of the nth multi-scale lane line. ; (1.5)。 3. The lane line detection method for complex road scenarios with multiple intersections according to claim 2, characterized in that, The point cloud volumetric feature network in step 2.2 consists of a voxel partitioning layer, an MLP layer, a convolutional layer, and a global max pooling layer. Step 2.2.1, the voxel division layer will Divide the region into vertical columnar regions with a grid size of P, and select features from the non-empty vertical columnar regions. ,in, express The characteristics of the i-th non-empty vertical columnar region, where M is the maximum number of non-empty vertical columnar regions; Step 2.2.2 After passing through an MLP layer, a convolutional layer, and a global max pooling layer, the nth lane line pseudo-image feature is output. .

4. The lane line detection method for complex road scenarios with multiple intersections according to claim 3, characterized in that, The lane line generative adversarial network in step 2.3 consists of a generator G and a discriminator D, and both the generator and the discriminator are composed of multi-layer convolutional modules, deep convolutional layers and residual blocks. Step 2.3.1 The input is processed by the generator G, and then sequentially through multiple convolutional modules, deep convolutional layers, and residual blocks to obtain the nth generation layer features. , After passing through a fully connected layer and normalization, the predicted nth multi-scale point cloud feature is output. , Voxel information representing the predicted lane line position; Step 2.3.2 and The input is processed by the discriminator D, and then sequentially through multiple convolutional modules, deep convolutional layers, and residual blocks to obtain the features of the nth discriminant layer. , After further processing by the Sigmoid activation function, the final output is... Probability of identifying a feature as a true point cloud feature ; Step 2.3.3: Construct the point cloud inversion loss of the generator G using equation (1.6). ; (1.6) In equation (1.6), express Expected value Indicates L1 loss, The weight hyperparameters representing the loss; Step 2.3.4: Construct the point cloud discrimination loss of the discriminator D using equation (1.7). ; (1.7) In equation (1.7), express Expected value; Step 2.3.5: Construct the weighted loss of the lane line generative adversarial network using equation (1.8). ; (1.8) In equation (1.8), and There are two weighting parameters used for balancing. and The impact.

5. The lane line detection method for complex road scenarios with multiple intersections according to claim 4, characterized in that, Step 2.4 includes: Step 2.4.1: Extract multi-scale lane line image features Optimal multi-scale point cloud features Projecting onto the same dimensional space, the query matrix can be obtained using equations (1.9), (1.10), and (1.11). Key matrix Value matrix : (1.9) (1.10) (1.11) In equations (1.9), (1.10), and (1.11), , , There are three projection matrices; Step 2.4.2: Generate attention weights between multimodal lane line features using equation (1.12). : (1.12) In equation (1.12), The dimensions of the key matrix and query matrix are represented. Indicates the activation function; Step 2.4.3, will and After dot product, the nth multimodal intermediate feature is obtained. ; Step 2.4.4, will and After splicing, the nth fused feature is obtained. .

6. The lane line detection method for complex road scenarios with multiple intersections according to claim 5, characterized in that, Step 2.5 includes: Step 2.5.1, Settings ,in, , , The step sizes are defined in the x, y, and z directions of the BEV space coordinate system, respectively. Cells are constructed using these step sizes to obtain a BEV space composed of several cells. Step 2.5.2: The viewpoint conversion module will fuse features. After sequential processing through convolutional layers, normalization, and the Sonftmax activation function, the probability distribution D of the nth lane line depth is obtained. n D n and After splicing, the nth deep fusion feature is obtained. ; Step 2.5.3, will Projecting the data into the BEV space, the three-dimensional point cloud coordinates of the lane lines are obtained. These three-dimensional coordinates are then assigned to the cells, resulting in the lane line features of the i-th cell in the BEV space. ; After passing through convolutional layers, pooling layers, normalization, and the ReLU function, the lane line feature map of the nth data sample is output. .

7. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store programs that support the processor in executing any of the lane detection methods of claims 1-6, and the processor is configured to execute the programs stored in the memory.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the lane detection method according to any one of claims 1-6.

Images