A lane line detection method based on multi-representation ensemble learning

Abstract

The application relates to a lane line detection method based on multi-representation integrated learning, comprising the following steps: constructing a lane line detection network, training the lane line detection network by using a vehicle front view image dataset, and obtaining a lane line detection model, wherein the lane line detection network architecture comprises a backbone network and a feature pyramid network connected in sequence, the feature pyramid network is connected with a segmentation branch and a curve fitting branch respectively, and an adaptive hybrid module is connected between the segmentation branch and the curve fitting branch; inputting an actual vehicle front view image into the lane line detection model, and outputting corresponding segmentation lane line detection results and curve fitting lane line detection results. Compared with the prior art, the lane line detection task is simultaneously represented as a segmentation task and a curve fitting task, a double-branch network structure is designed, the advantages of lane line segmentation representation and curve representation are combined, and the accuracy of lane line detection can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a lane line detection method based on multi-representation ensemble learning. BACKGROUND

[0002] Lane line information is the basis for lane keeping, yaw warning and other functions, and also plays an auxiliary role in vehicle positioning. For an automatic driving system, a lane line detection system with high precision and good real-time performance is crucial for the vehicle to make reasonable decision control schemes while driving. The lane line detection system mainly consists of two parts: vehicle-mounted sensor hardware and detection algorithm software. The vehicle-mounted sensor mainly includes a camera, a laser radar, etc., among which the vehicle-mounted camera is the most widely used sensor in current automatic driving due to its low cost and rich information. The detection algorithm extracts the features of the lane line from the environmental data measured by the vehicle-mounted sensor, and predicts the accurate position of the lane line through these feature information. In addition, the lane line detection algorithm based on vision has also become a research hotspot in current automatic driving, in cooperation with the sensor.

[0003] However, lane line detection is actually a quite challenging task. First, there are many lane line shapes, including different colors, solid and dashed lines, curvatures, etc., and there are also merging and separating situations at ramps, intersections, etc. Second, the environmental conditions for lane line detection are also quite complex, such as poor visual conditions caused by extreme light and weather, occlusions caused by vehicles and obstacles, and the situation of missing lane lines due to aging of some roads.

[0004] Over a long period of time, a variety of lane line representations have been used for lane line detection algorithms: segmentation representation, curve representation, point representation, etc. Segmentation representation classifies whether each pixel in the image belongs to a lane line pixel, and finally detects the lane line in the image in a pixel-by-pixel labeling manner; curve representation models the lane line as a curve, and fits the curve parameters according to the extracted lane line features to detect the lane line in the image in the form of curve parameters; point representation has two ways, one of which is to preset lane line priors, and to classify whether the pixel belongs to the lane line pixel on the lane line priors, and the other of which is not to preset lane line priors, but to use row-by-row detection to classify whether the pixel belongs to the lane line pixel, both of which detect the lane line in the image in the form of lane line points.

[0005] However, in practical applications, the pixel-by-pixel prediction of segmentation representation easily ignores the inherent shape priors of lane lines or the strong correlation between lanes, and is difficult to deal with situations such as lane line occlusion and missing; curve representation is highly abstract, and has the problems of training difficulty and poor performance; the point representation with preset lane priors must design heuristic lane anchors, and highly depends on the statistics of the data set, while the point representation with row-by-row detection has a sharp decline in prediction performance in situations with long distances or large curvature changes. SUMMARY

[0006] The present application aims to overcome the defects of the prior art and provides a lane line detection method based on multi-representation integrated learning, which simultaneously uses segmentation representation and curve representation for lane line detection and realizes information interaction of lane line information of the two representation methods to effectively improve the accuracy of lane line detection.

[0007] The purpose of the present application can be achieved by the following technical scheme: a lane line detection method based on multi-representation integrated learning, comprising the following steps:

[0008] S1, constructing a lane line detection network and training the lane line detection network using a vehicle front view image dataset to obtain a lane line detection model;

[0009] The lane line detection network architecture comprises a backbone network and a feature pyramid network connected in sequence, the feature pyramid network is connected with a segmentation branch and a curve fitting branch respectively, and the segmentation branch and the curve fitting branch are connected with an adaptive mixing module;

[0010] S2, inputting an actual vehicle front view image into the lane line detection model to output corresponding segmentation lane line detection results and curve fitting lane line detection results.

[0011] Further, the specific process of training the lane line detection network in step S1 is:

[0012] S11, inputting a vehicle front view image into the backbone network to extract multi-level image features;

[0013] S12, inputting the extracted multi-level image features into the feature pyramid network to realize fusion of high-level features and low-level features and obtain fused image features;

[0014] S13, inputting the fused image features into the curve fitting branch and the segmentation branch respectively, obtaining an attention map and a query vector containing curve parameter information in the curve fitting branch;

[0015] S14, splicing the fused image features and the attention map in the segmentation branch;

[0016] S15, inputting the query vector, the spliced fused image features and the attention map into the adaptive mixing module to realize information interaction of the segmentation branch and the curve fitting branch, obtaining an interactive query vector and an interactive image feature;

[0017] S16, in the segmentation branch, the interactive image features are input into a segmentation decoder to obtain a lane line prediction in a segmentation form; in the curve fitting branch, the interactive query vectors are input into a multi-layer perception and a full connection layer to obtain a lane line prediction in a curve form, wherein the segmentation lane line prediction and the curve fitting lane line prediction are respectively optimized by using corresponding loss functions.

[0018] Further, the multi-level image features in the step S11 are four-level features output by the first four layers of the backbone network ResNet, and the down-sampling rates of the four-level features are 4, 8, 16 and 32 in sequence.

[0019] Further, the feature pyramid network in the step S12 is a network taking the four-level features as input and outputting a fusion image feature with a down-sampling rate of 8;

[0020] The specific working process of the feature pyramid network is as follows: the feature fusion is divided into two stages. In the first stage, the high-level features are subjected to 1x1 convolution, batch normalization processing and nonlinear activation, and then are bilinearly up-sampled and added to the low-level features in sequence.

[0021] In the second stage, the features at all levels are first subjected to 3x3 convolution, batch normalization processing and nonlinear activation, then, the 4x down-sampled features are down-sampled to 8x down-sampling rate by 3x3 convolution with a step of 2, the 16x and 32x down-sampled features are bilinearly interpolated to 8x down-sampling rate, and the features at all levels are spliced, subjected to 3x3 convolution, batch normalization processing and nonlinear activation to obtain the fusion image feature.

[0022] Further, the curve fitting branch in the step S13 is a Transformer structure comprising a complete encoder and a decoder, wherein the encoding layer and the decoding layer are both 2 layers.

[0023] The specific working process of the curve fitting branch in the step S13 is as follows: the fusion image features are unfolded into a sequence, added to the position encoding and input into the encoder.

[0024] In the encoder, the sequence of the fusion features is linearly projected to obtain a key vector and a value vector, the feature sequence itself is taken as a query vector to perform self-attention calculation to obtain an attention map.

[0025] The attention map, the position encoding and an initial query vector q initialized as zero are input into the decoder. In the decoder, the q is first linearly projected to obtain a key vector and a value vector, the q itself is taken as a query vector to perform self-attention calculation, then the attention map output by the encoder is linearly projected to obtain a key vector, the attention map is taken as a value vector, and the q is taken as a query vector to perform mutual attention calculation to obtain a query vector containing curve parameter information.

[0026] Further, the step S15 is to perform information interaction between the image feature X and the query vector q, and the working process of the adaptive mixing module is as follows:

[0027] The query vector q is first subjected to linear operation to obtain a channel adaptive mixing weight M c and a spatial adaptive mixing weight M s , and the image feature X is subjected to channel adaptive mixing and spatial adaptive mixing in sequence to obtain the interactive image feature X ′ ; the mixing process is to perform matrix multiplication between the image feature and the corresponding adaptive mixing weight, and then sequentially perform layer normalization processing and nonlinear activation; the interactive feature X ′ is subjected to two projection transformations and then connected with the original query vector q to obtain the interactive query vector q ′ .

[0028] Further, the segmentation decoder in the step S16 includes a coarse decoding branch and a fine decoding branch, in the decoding process, the coarse decoding branch uses 1*1 convolution and bilinear interpolation upsampling for decoding to obtain a preliminary and coarse segmentation result; the fine decoding branch uses multiple upsampling modules composed of deconvolution and non-bottleneck layers for decoding, and the result is added to the coarse decoding branch to complete the refinement of the segmentation result, each channel of the segmentation feature map corresponds to predicting a lane line instance;

[0029] The multi-layer perception in the step S16 is composed of three linear operations and nonlinear activation, and the fully connected layer in the step S16 is composed of one linear operation.

[0030] Further, the loss function corresponding to the segmentation lane line prediction in the step S16 is composed of two parts: a segmentation loss L seg and a lane line existence loss L exist .

[0031] The loss function corresponding to the curve fitting lane line prediction in the step S16 is a two-part matching loss, which is composed of three parts: a classification loss L label of whether the curve is a lane line, a curve parameter loss L para , and a curve sampling point loss L sample .

[0032] The loss function of the task training in the step S16 is the weighted sum of the above losses:

[0033] LOSS=w seg L seg +w exist L exist +w label L label +w para Lpara +w sample L sample

[0034] where L seg is the segmentation loss, L exist is the lane existence loss, L label is the curve classification loss, L para is the curve parameter loss, and L sample is the curve sample loss. γ where γ label =seg, exist, label, para, sample correspond to the weights of the above losses.

[0035] Further, the curve fitting branch in step S16 predicts a fixed number of curve instances, outputs a binary classification of whether the curve is a lane line and curve parameters, and uses the Hungarian algorithm to match the predictions and ground truths, the matching cost function includes three parts: curve classification cost C para , curve parameter cost C sample , and curve sample cost C label , and the total cost is the weighted sum of the above three costs:

[0036] COST = α label C para + α para C sample + α sample C

[0037] where curve classification cost C label uses cross-entropy loss, curve parameter cost C para and curve sample cost C sample use L1 loss, and α j (j = label, para, sample) correspond to the weights of each cost.

[0038] Further, the curve parameters are the coordinates of the four control points of the third-order Bezier curve, and the curve sample points are the sampling points B(t) obtained by t-sampling the Bezier curve:

[0039]

[0040]

[0041] where P i is the i-th control point, and b i,3 (t) is the third-order Bezier coefficient.

[0042] Compared with the prior art, the present application has the following advantages:

[0043] I. The present application considers the deficiency of single representation method, so a double branch model structure is designed from the representation method of lane line detection, including a backbone network and a feature pyramid network connected in turn, the feature pyramid network is connected with a segmentation branch and a curve fitting branch respectively, an adaptive mixing module is connected between the segmentation branch and the curve fitting branch, lane line detection is performed by the two branches in segmentation representation and curve representation respectively, and the adaptive mixing module is used to realize the interaction of lane line information of the two representation methods. Therefore, lane line detection is performed in segmentation and curve two representation methods in a single model, and the information interaction of the two representations is completed, which can effectively improve the accuracy of lane line detection in segmentation and curve two representation methods.

[0044] II. The present application considers that lane lines have high-level semantics, and high-level features are very important for the prediction of the existence of lane line instances, and lane lines also have specific local patterns, which require detailed low-level features to accurately locate. Therefore, the present application designs a feature pyramid network to realize the fusion of high-level features and low-level features to ensure the accuracy of subsequent lane detection.

[0045] III. The present application uses an adaptive mixing module to interact the information of image features X and query vector q, to realize the information interaction of the segmentation branch and the curve fitting branch, then inputs the interacted image features into a segmentation decoder to obtain lane line prediction in segmentation representation, and inputs the interacted query vector into a multilayer perceptron and a fully connected layer to obtain lane line prediction in curve representation, which can greatly improve the lane line detection performance. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 It is a method flowchart of the present application;

[0047] Figure 2 It is a lane line detection model architecture diagram in the embodiment. DETAILED DESCRIPTION

[0048] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0049] EMBODIMENT

[0050] As shown in the figure, a lane line detection method based on multi-representation integrated learning includes the following steps: Figure 1

[0051] S1, construct a lane line detection network, and train the lane line detection network using vehicle front view image data set to obtain a lane line detection model;

[0052] ​The lane line detection network architecture comprises a backbone network and a feature pyramid network connected in sequence, the feature pyramid network is connected with a segmentation branch and a curve fitting branch respectively, and an adaptive hybrid module is connected between the segmentation branch and the curve fitting branch.

[0053] S2, input the actual vehicle front view image into the lane line detection model, and output corresponding segmentation lane line detection results and curve fitting lane line detection results.

[0054] The embodiment applies the above technical solutions to construct a lane line detection model as shown in Figure 2 The main content includes:

[0055] 1) using a monocular RGB camera to collect a vehicle front view image, inputting the image into a backbone network to extract multi-level image features, wherein the multi-level image features are four-level features output by the first four layers of the backbone network ResNet, and the down-sampling rates of the four-level features are 4, 8, 16 and 32 in sequence.

[0056] 2) inputting the multi-level image features extracted in the backbone network into a feature pyramid network to realize fusion of high-level and low-level features, adding the first-stage high-level features after convolution and up-sampling to the low-level features, and connecting the second-stage high-level features after convolution to the low-level features;

[0057] The feature pyramid network is a network taking the four-level features obtained in step 1) as input and outputting fused image features with a down-sampling rate of 8. The feature fusion is divided into two stages. In the first stage, the high-level features are subjected to 1x1 convolution, batch normalization processing and nonlinear activation, and then bilinear interpolation up-sampling is performed, and the high-level features are added to the low-level features level by level. In the second stage, each level of features is first subjected to 3x3 convolution, batch normalization processing and nonlinear activation, then the 4x down-sampling features are down-sampled to 8x down-sampling rate by 3x3 convolution with a step of 2, and the 16x and 32x down-sampling features are bilinearly interpolated to 8x down-sampling rate. After the features of each level are spliced, 3x3 convolution, batch normalization processing and nonlinear activation are performed to obtain the fused image features.

[0058] 3) the fused features in step 2) are input into the curve fitting branch and the segmentation branch respectively. The fused features are expanded into a sequence and input into an encoder together with position encoding for self-attention calculation to obtain an attention map, the attention map is input into a decoder together with position encoding, and an initial query vector is also input into the decoder, and mutual attention calculation is performed in the decoder to learn the query vector of the lane line object.

[0059] Specifically, the curve fitting branch is a Transformer structure containing a complete encoder and decoder, where both the encoding layer and the decoding layer are 2 layers. The fusion features are unfolded into a sequence, and after adding the position encoding, they are input into the encoder. In the encoder, the sequence of the fusion features is linearly projected to obtain key vectors and value vectors, and the feature sequence itself is taken as a query vector to perform self-attention calculation to obtain an attention map. The attention map, the position encoding, and an initial query vector q initialized to zero are input into the decoder. In the decoder, the q is first linearly projected to obtain key vectors and value vectors, and the q itself is taken as a query vector to perform self-attention calculation. Then, the key vectors of the attention map output by the encoder are linearly projected, the attention map is taken as a value vector, and the q is taken as a query vector to perform mutual attention calculation to obtain a query vector containing curve parameter information.

[0060] 4) The fusion features of step 2) and the attention map of step 3) are spliced in the segmentation branch;

[0061] 5) The query vector in step 3), the spliced image fusion features and the attention map in step 4) are input into the adaptive mixing module for information interaction. In the module, the query vector is first subjected to linear operation to obtain channel adaptive mixing weights and spatial adaptive mixing weights, and the attention features are first subjected to channel adaptive mixing and then subjected to spatial adaptive mixing to obtain the image features after interaction. The mixing process is that the image features are multiplied by the corresponding adaptive mixing weights, and then layer normalization processing and nonlinear activation are sequentially performed. After the features after interaction are subjected to two projection transformations, the residual connection is performed with the query vector in step 3) to obtain the query vector after interaction.

[0062] Specifically, the adaptive mixing module performs information interaction of the image features X and the query vector q. In the module, the query vector q is first subjected to linear operation to obtain channel adaptive mixing weights M c and spatial adaptive mixing weights M s , and the image features X are subjected to channel adaptive mixing and spatial adaptive mixing in sequence to obtain the image features X ′ after interaction. The mixing process is that the image features are multiplied by the corresponding adaptive mixing weights, and then layer normalization processing and nonlinear activation are sequentially performed. After the features X ′ after interaction are subjected to two projection transformations, the residual connection is performed with the original query vector q to obtain the query vector q ′ after interaction. The expression of the calculation process is as follows:

[0063] M c = Linear(q) (1)

[0064] M s = Linear(q) (2)

[0065] X1 = ReLU(Layernorm(XM c )) (3)

[0066]

[0067] q ′ = q + Proj(Proj(X ′ )) (5)

[0068] 6) In the segmentation branch, the interactive image features obtained in step 5) are input into the segmentation decoder. The decoder is divided into coarse and fine branches. The coarse decoding branch uses convolution and bilinear interpolation upsampling to obtain a preliminary and rough upsampling result. The fine decoding branch uses deconvolution and non-bottleneck module upsampling and adds the result of the coarse decoding branch to complete the refinement of the upsampling. In the curve branch, the query vector obtained in step 5) is input into the fully connected layer and the multilayer perceptron to obtain the classification of whether the curve is a lane line and the curve parameters, respectively. The multilayer perceptron is composed of cubic linear operation and nonlinear activation, and the fully connected layer is composed of linear operation. The curve classification is a binary classification of whether it is a lane line. The curve parameters are the coordinates of the four control points of the cubic Bezier curve.

[0069] 7) The lane line detection model composed of steps 1)-6) is trained using an open source dataset. During the training process, steps 1)-6) are repeated, and the segmentation branch and the curve branch are optimized using the loss function respectively until the model converges.

[0070] Specifically, the segmentation representation loss function is composed of two parts: the segmentation loss L seg and the lane line existence loss L exist . The curve representation loss is a binary matching loss, which is composed of three parts: the classification loss L label of whether the curve is a lane line, the curve parameter loss L para , and the curve sampling point loss L sample .

[0071] The loss function for task training is the weighted sum of the above losses:

[0072] LOSS = w seg L seg + w exist L exist + w label L label + w para L para + w sample L sample (6)

[0073] where the segmentation loss uses pixel-wise cross-entropy loss, the lane existence loss, the curve classification loss use cross-entropy loss, and the curve parameter loss and the curve sample point loss use L1 loss.w γ (γ = seg, exist, label, para, sample) are the weights of the above losses, in this embodiment, the weights are set to 1, 0.1, 1, 1, 1, respectively.

[0074] For bipartite matching, the curve branch prediction fixes the number of curve instances, outputs the binary classification of whether the curve is a lane line and the curve parameters, and uses the Hungarian algorithm to match the prediction and the true value. The matching cost function includes three parts: curve classification cost C label , curve parameter cost C para , and curve sample point cost C sample , and the total cost is the weighted sum of the above three costs:

[0075] COST = α label C label + α para C para + α sample C sample (7)

[0076] where the curve classification cost uses cross-entropy loss, the curve parameter cost and the curve sample point cost use L1 loss, and α j (j = label, para, sample) are the weights of each cost, which are all set to 1 in this embodiment.

[0077] For curve sample points, t-sampling is performed on the Bezier curve to obtain the sample points B(t). A third-order Bezier curve is a curve defined by four control points:

[0078]

[0079] where P i is the i-th control point, and b i,3 (t) is the third-order Bezier coefficient:

[0080]

[0081] In this embodiment, 50 points are sampled at equal intervals in the range of 0≤t≤1.

[0082] 8) After the training is completed, the model is used for lane line detection tasks, and the lane line position in the image can be obtained by inputting the road RGB image to be detected;

[0083] After the above eight steps are completed, the lane line detection is completed.

[0084] In conclusion, the technical scheme takes two classical representation methods in lane line detection algorithm, i.e., segmentation representation and curve representation, as the breakthrough point, proposes a brand-new multi-representation integrated learning method, simultaneously performs lane line detection in a single model by using the two representation methods, and completes information interaction of the two representations, so that the detection accuracy of lane line detection by using the two representation methods can be simultaneously improved.

Claims

1. A lane line detection method based on multi-representation ensemble learning, characterized in that, The method comprises the following steps: S1, constructing a lane line detection network and training the lane line detection network using a vehicle front view image dataset to obtain a lane line detection model; The lane line detection network architecture comprises a backbone network and a feature pyramid network connected in sequence, the feature pyramid network is connected with a segmentation branch and a curve fitting branch respectively, and the segmentation branch and the curve fitting branch are connected with an adaptive mixing module; S2, inputting an actual vehicle front view image into the lane line detection model to output corresponding segmentation lane line detection results and curve fitting lane line detection results; The feature pyramid network is a network taking four-level features as input and outputting fused image features with a down-sampling rate of 8; The curve fitting branch specifically comprises a Transformer structure comprising a complete encoder and a decoder, wherein the encoding layer and the decoding layer are both 2 layers; The adaptive mixing module specifically performs information interaction of image features and query vectors The working process of the adaptive mixing module is as follows: Query vector First, the channel adaptive mixing weight is obtained through linear operation And the spatial adaptive mixing weight Image features The interactive image features are obtained through channel adaptive mixing and spatial adaptive mixing in sequence ; The mixing process is matrix multiplication of image features and corresponding adaptive mixing weights, followed by layer normalization processing and nonlinear activation in sequence; the interactive feature After two projection transformations, the original query vector The residual connection is performed to obtain the interactive query vector . 2.The lane line detection method based on multi-representation ensemble learning according to claim 1, characterized in that, The specific process of training the lane line detection network in step S1 is as follows: S11, inputting the vehicle front view image into the backbone network to extract multi-level image features; S12, inputting the extracted multi-level image features into the feature pyramid network to realize fusion of high-level features and low-level features and obtain fused image features; S13, inputting the fused image features into the curve fitting branch and the segmentation branch respectively, obtaining an attention map and a query vector containing curve parameter information in the curve fitting branch; S14, splicing the fused image features and the attention map in the segmentation branch; S15, inputting the query vector, the spliced fused image features and the attention map into the adaptive mixing module to realize information interaction between the segmentation branch and the curve fitting branch, obtaining an interactive query vector and an interactive image feature; S16, in the segmentation branch, inputting the interactive image feature into a segmentation decoder to obtain a segmentation-represented lane line prediction; In the curve fitting branch, inputting the interactive query vector into a multi-layer perceptron and a fully connected layer to obtain a curve-represented lane line prediction, wherein the segmentation lane line prediction and the curve fitting lane line prediction are respectively optimized using corresponding loss functions. 3.The lane line detection method based on multi-representation ensemble learning according to claim 2, characterized in that, The multi-level image features in step S11 are specifically four-level features with down-sampling rates of 4, 8, 16 and 32 output by the first four layers of the backbone network ResNet.

4. The lane line detection method based on multi-representation ensemble learning according to claim 3, characterized in that, The specific working process of the feature pyramid network is as follows: feature fusion is divided into two stages, in the first stage, high-level features are subjected to 1×1 convolution, batch normalization processing and nonlinear activation, and then bilinear interpolation up-sampling is performed, and the high-level features are added to low-level features level by level; In the second stage, each level of features is first subjected to 3×3 convolution, batch normalization processing and nonlinear activation, then 4 times down-sampling features are down-sampled to 8 times down-sampling rate by 3×3 convolution with a step of 2, 16 and 32 times down-sampling features are bilinearly interpolated to 8 times down-sampling rate, and after the features at each level are spliced, 3×3 convolution, batch normalization processing and nonlinear activation are performed to obtain fused image features.

5. The lane line detection method based on multi-representation ensemble learning according to claim 2, characterized in that, The specific working process of the curve fitting branch in step S13 is as follows: the fused image features are unfolded into a sequence, added to position encoding and input into the encoder; In the encoder, the sequence of fused features is linearly projected to obtain a key vector and a value vector, and the self-attention calculation is performed on the feature sequence itself as a query vector to obtain an attention graph; An attention map, position encoding and an initial query vector initialized with zero An input decoder, in which firstly the Linear projection to get key vector and value vector, Self-attention calculation with itself as query vector, and linear projection of the attention map output by the encoder to get key vector, the attention map as value vector, Mutual attention calculation with the query vector to get the query vector containing curve parameter information. 6.The lane line detection method based on multi-representation ensemble learning according to claim 2, characterized in that, The segmentation decoder in the step S16 includes a coarse decoding branch and a fine decoding branch, and in the decoding process, the coarse decoding branch uses 1*1 convolution and bilinear interpolation upsampling for decoding to obtain a preliminary and coarse segmentation result; The fine decoding branch uses a plurality of upsampling modules composed of deconvolution and non-bottleneck layers for decoding, and the result is added to the coarse decoding branch to complete the refinement of the segmentation result, and each channel of the segmentation feature map corresponds to predicting a lane line instance; The multilayer perceptron in the step S16 is composed of cubic linear operation and nonlinear activation, and the fully connected layer in the step S16 is composed of linear operation.

7. The lane line detection method based on multi-representation ensemble learning according to claim 6, characterized in that, The loss function corresponding to the lane line segmentation prediction in the step S16 is composed of two parts: segmentation loss and lane line existence loss ; The loss function corresponding to the curve fitting lane line prediction in the step S16 is a two-part matching loss, and the two-part matching loss is composed of three parts: a classification loss of whether the curve is a lane line , a curve parameter loss , and a curve sampling point loss . The loss function of the task training in the step S16 is the weighted sum of the above losses: ， where segmentation loss using pixel-wise cross-entropy loss, lane existence loss and curve classification loss using cross-entropy loss, curve parameter loss and curve sampling point loss using L1 loss, corresponding to the weights of the above losses, gamma = seg, exist, label, para, sample . 8.The lane line detection method based on multi-representation ensemble learning according to claim 7, characterized in that, The curve fitting branch in the step S16 predicts a fixed number of curve instances, outputs a binary classification of whether the curve is a lane line and curve parameters, and uses the Hungarian algorithm to match the predicted and true values, and the matching cost function includes three parts: curve classification cost , curve parameter cost , and curve sampling point cost , and the total cost is a weighted sum of the above three costs: ， wherein the curve classification cost with cross-entropy loss, the curve parameter cost and the curve sample point cost using L1 loss, corresponding to the weights of the costs, j = label, para, sample . 9.The lane line detection method based on multi-representation ensemble learning according to any one of claims 7-8, characterized in that, The curve parameters are four control point coordinates of a third-order Bezier curve, and the curve sampling points are obtained by sampling the Bezier curve sampling to obtain sampling points : ， ， wherein is the first control point, is a third order Bezier coefficient.

Images