A parallel multi-scale feature aggregation lane detection method based on key point estimation

Abstract

The present invention relates to the fields of autonomous driving and computer vision, and discloses a lane line detection method based on key point estimation and parallel multi-scale feature aggregation, comprising the following steps: 1) obtaining a lane line image data set for training a lane line detection model; 2) performing data preprocessing on the lane line image data set, uniformly scaling the image size and normalizing; 3) constructing a parallel multi-scale feature aggregation lane line detection model based on key point estimation; 4) feeding the lane line image data set into the detection model for training; 5) inputting an image or video to be tested into the trained and converged detection model, and the model performs lane line detection and outputs lane line detection information of the image or video. While taking into account the lane line detection accuracy and detection speed, the present invention can also adapt to any structure and any number of road scenes, and has high robustness and practicality.

Description

Technical Field

[0001] The present invention relates to the fields of autonomous driving and computer vision, and in particular to a parallel multi-scale feature aggregation lane line detection method based on key point estimation. Background Art

[0002] Fully automatic driving mainly includes three core modules: perception, decision-making and control. Among them, perception is to obtain comprehensive environmental information around the car through various sensors and control modules, which is the basis of decision-making and control. Lane detection is a basic environmental perception task, and its detection speed and accuracy determine the performance of the automatic driving system. Due to factors such as vehicle occlusion, wear of road signs, and dark light environment at night, lane detection still faces great challenges in practical applications. Therefore, it is of practical significance to study lane detection methods that are adaptable to a wide range of scenarios, have strong robustness, and have high detection accuracy and efficiency.

[0003] Traditional lane detection methods usually rely on manually extracted low-level features or solidified mathematical models. The manually designed features and the strong a priori nature of the mathematical models are very sensitive to environmental changes. Therefore, traditional methods have problems such as low detection accuracy and weak model generalization ability in complex and changeable real-world scenarios.

[0004] Lane detection methods based on deep learning can achieve pixel-level detection with their powerful representation learning ability and massive data resources. At present, deep learning detection methods have achieved good performance in most scenarios. These methods can be divided into segmentation-based methods, detection-based methods and parameter prediction methods. Segmentation-based methods focus on network structure, and most of their network structures adopt a high-to-low and low-to-high tandem encoding-decoding structure. When facing complex scenes with weak appearance (such as severe vehicle occlusion, shadows, blurred markings, etc.), the tandem structure often ignores the geometric prior and correlation of the lane itself, making it difficult to extract effective discriminative features, resulting in reduced detection accuracy. Detection-based methods focus on the prediction output of the network, which is not limited to the structure of the network, but generates a large number of parameterized predictions, which increases the computational cost of the model. In addition, in order to geometrically shape the parameterized results, a post-processing step needs to be inserted, which further reduces the detection efficiency of the algorithm. The parameter prediction method directly predicts the parameter equation to represent the geometric line type of the lane. The limitation is that it can only detect fixed structures and a fixed number of lanes. Summary of the invention

[0005] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a parallel multi-scale feature aggregation lane line detection method based on key point estimation.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A parallel multi-scale feature aggregation lane detection method based on key point estimation, the method comprising the following steps:

[0008] S1. Obtain a lane line image dataset for training a lane line detection model;

[0009] S2, preprocessing the obtained lane line image dataset, uniformly scaling the image size and normalizing it, and applying multiple data enhancement strategies;

[0010] S3. Construct a parallel multi-scale feature aggregation lane detection model based on key point estimation;

[0011] The detection model includes an input image initialization module, a parallel multi-scale feature aggregation network, a global key point detection module, a key point association clustering module, and an image output module; after the input image passes through the input image initialization module, it is fed into the parallel multi-scale feature aggregation network for feature extraction, the global key point detection module accurately locates the global lane line key points based on the feature map output by the network, and then the key point association clustering module integrates the global key point detection results into the local lane curve geometry modeling based on the embedding vector generated by the network for each key point, and finally, the lane line prediction results of the visualization model are visualized by the output module;

[0012] After data preprocessing of the input image initialization module: the input image size is compressed to 32×64 resolution, and the image channel dimension is expanded from 3 to 32;

[0013] S4, feeding the lane line image dataset into the detection model for training, adjusting the model hyperparameters, and training the model until the loss is fully converged to obtain a trained detection model;

[0014] S5. Input the image or video to be tested into the trained and converged detection model. The model performs lane line detection and outputs lane line detection information of the image or video.

[0015] Furthermore, the lane line image dataset in S1 includes TuSimple and CULane lane line datasets;

[0016] The TuSimple dataset is mainly composed of highway scenes at different times of the day under good and moderate weather conditions, including 3626 training images and 2782 test images. Accuracy is the main evaluation indicator of the dataset.

[0017] The CULane dataset has 88,880 training images, 9,675 verification images, and 34,680 test images, covering nine different road scenes (such as congestion, shadow, night, glare, etc.), and the F1 score is its main evaluation indicator.

[0018] Furthermore, the lane line image dataset in S2 is preprocessed by adjusting the image size to a uniform resolution of 256×512 and normalizing the image RGB value to 0-1.

[0019] Furthermore, the parallel multi-scale feature aggregation network in S3 can integrate the global spatial information of multiple scale feature maps in parallel and enrich the lane feature information; the network architecture of the parallel multi-scale feature aggregation network consists of an encoder and a decoder, which are connected in parallel.

[0020] Furthermore, the parallel multi-scale feature aggregation network in S3 integrates a dual-headed attention feature fusion upsampling module (DAFU). DAFU aggregates global information from two different receptive domains, interpolation coarsening sampling and transposed convolutional refinement sampling, along the channel dimension through attention feature fusion, and can accurately restore low-resolution feature maps to pixel-level predictions in parallel at multiple spatial scales; DAFU consists of a transposed convolution refinement branch, an interpolation sampling coarsening branch, and an attention feature fusion module, which can be expressed as:

[0021]

[0022] Among them, T(·) represents transposed convolution or deconvolution, B(·) represents interpolation upsampling, M(·,·) represents attention feature fusion, δ is the sigmoid function, and x is the input feature map.

[0023] Furthermore, the global key point detection module in S3 predicts a set of key point probability heat maps and a set of key point position offset vector heat maps based on a parallel multi-scale feature aggregation network, accurately locates the position of lane key points, and ensures the geometric accuracy of the final output lane curve; wherein the probability heat map is used to model the probability space of all target key points and predict the probability of lane line key points of each grid in the image; the offset heat map predicts the position offset vector of the lane line key points;

[0024] The global key point detection module applies the balanced focus loss function and the mean square error loss function to train the network for the probability heat map output and the offset heat map output, respectively. In the training data, each lane curve is marked as a set of ordered key point sets l m ={k i |i=1,2,…,n}, where l m Indicates that the mth lane contains n key points, ki represents the i-th key point; the probability heat map loss and position offset loss are expressed as:

[0025]

[0026]

[0027] Among them, L kp is the key point probability heat map loss, L position is the key point position offset loss, P i is the probability score of the key point at the i-th grid position in the heat map, N k is the total number of key points in the current image, g i represents the true value of positive and negative samples, α is the balance factor, which balances the positive and negative sample losses in the training process, γ is an adjustable hyperparameter, which controls the penalty for negative samples, and P i_x and P i_y are the horizontal and vertical axis offset predictions, G i_x and G i_y is the true value.

[0028] Furthermore, the key point association clustering module in S3 further subdivides the global detection key point results of the previous part to determine the lane category to which they belong. For this part, the network generates a feature embedding vector for all detected key points, and trains the network to complete the clustering task by applying an association embedding clustering loss function to the key point feature embedding vector. The key point association clustering loss function of this part can be expressed as:

[0029]

[0030]

[0031]

[0032] L GP =L assemble +L separate

[0033] in, is a keypoint embedding of lane l, K l is the total number of keypoint embeddings for lane l, N represents the total number of lanes in the image, and μ l represents the cluster center of l (average embedding), ||·||2 represents the L2 distance, Δ represents the distance threshold between cluster centers, and L assemble is the ensemble loss, which makes the keypoint embeddings of the same lane category as close as possible to the cluster center μ of the lane embedding. l ; L separateIt is the separation loss, which separates the cluster centers of different lane key points embedding so that the key points of different lane categories are far away from each other. The total clustering loss is the sum of the set loss and the separation loss.

[0034] Furthermore, the key point association clustering module, after the model training converges, uses a distance-based clustering technique to segment each lane line key point through an iterative process. The specific operations are as follows:

[0035] 1) First, set the embedding feature distance threshold. To avoid selecting an outlier as the threshold, use mean shift to make it closer to the cluster center.

[0036] 2) Select any lane keypoint embedding and assign a lane line label to it. The value of the keypoint embedding is used as the cluster center of the lane line embedding;

[0037] 3) Select an embedding from the remaining key point embeddings and calculate the feature distance (L2 distance) between it and . If the result is less than or equal to the distance threshold, assign the same label to the embedding and take the average of the sum as the cluster center of the new lane line embedding. Otherwise, continue to judge the next point.

[0038] 4) Repeat step 3, traverse all remaining key point embeddings, and find all key point embeddings of lane lines;

[0039] 5) The remaining keypoint embeddings that are not assigned lane labels are iterated again, and steps 2, 3, and 4 are repeated until all keypoint embeddings are assigned a lane label.

[0040] Furthermore, the image output module in S3 visually marks the coordinates of the key points after refinement and clustering in the original input image, and distinguishes each lane key point with a different color mark, and outputs the final prediction result.

[0041] Furthermore, in S4, the image batch size is set to 8, and the model is trained for 300 batches using the Adam optimizer with a learning rate of 0.0001 and the “poly” learning rate decay strategy, and the α, γ, Δ, δ in the loss function are adjusted. i The model loss gradually converges to the minimum value and saves the optimal model.

[0042] Beneficial effects of the present invention:

[0043] 1. This paper abstracts lane lines into a set of discrete key points and proposes a parallel multi-scale feature aggregation lane line detection method based on key point estimation. Compared with previous detection methods, this method directly predicts the key points of the lane from the input image or video accurately, which not only eliminates the target detection box and complicated post-processing requirements, but also reduces the influence of background pixels, thereby simplifying the network output, and is suitable for processing any number of lane scenes.

[0044] 2. The parallel multi-scale feature aggregation network described in the present invention, by integrating the global spatial information of multiple scale feature maps and utilizing local clues at other locations to enrich global features, can capture recognizable lane line feature information in weak appearance scenes such as occlusion, road wear, shadows and dark light, thereby improving the detection performance of the model.

[0045] 3. The dual-headed attention feature fusion upsampling module (DAFU) described in the present invention can realize the fine restoration of low-resolution feature maps to higher-resolution representations, help the network accurately identify and detect targets under extreme scale changes, and effectively integrate the spatial information of deep features.

[0046] 4. The parallel multi-scale feature aggregation lane line detection method based on key point estimation described in the present invention has a detection accuracy and F1 value of 96.82% and 75.6% on the Tusimple and CULane lane data sets respectively, and a real-time detection rate of 25ms, which is superior to the current mainstream lane detection method and has high practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings.

[0048] Figure 1 It is a diagram of the overall architecture of the model in the embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of the structure of an input image initialization module in an embodiment of the present invention;

[0050] Figure 3 is a schematic diagram of a parallel multi-scale feature aggregation network structure in an embodiment of the present invention;

[0051] Figure 4 2 is a schematic diagram of the structure of a dual-head attention feature fusion upsampling module in an embodiment of the present invention;

[0052] Figure 5 is a visualization result diagram of lane line detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, a parallel multi-scale feature aggregation lane line detection method based on key point estimation is characterized in that the method comprises the following steps:

[0055] S1. Obtain a lane line image dataset for training a lane line detection model;

[0056] The lane line image dataset in S1 includes TuSimple and CULane lane line datasets;

[0057] The TuSimple dataset is mainly composed of highway scenes at different times of the day under good and moderate weather conditions, including 3626 training images and 2782 test images. Accuracy is the main evaluation indicator of the dataset.

[0058] The CULane dataset has 88,880 training images, 9,675 verification images, and 34,680 test images, covering nine different road scenes (such as congestion, shadow, night, glare, etc.), and the F1 score is its main evaluation indicator.

[0059] S2, preprocessing the obtained lane line image dataset, uniformly scaling the image size and normalizing it, and applying multiple data enhancement strategies;

[0060] The data preprocessing method of the lane line image dataset in S2 is: the image size is uniformly adjusted to a resolution of 256×512, and the image RGB value is normalized to 0-1.

[0061] S3. Construct a parallel multi-scale feature aggregation lane detection model based on key point estimation;

[0062] like Figure 2As shown in the figure, the detection model includes an input image initialization module, a parallel multi-scale feature aggregation network, a global key point detection module, a key point association clustering module, and an image output module; after the input image passes through the input image initialization module, it is fed into the parallel multi-scale feature aggregation network for feature extraction, the global key point detection module accurately locates the global lane line key points based on the feature map output by the network, and then the key point association clustering module integrates the global key point detection results into the local lane curve geometry modeling by applying the association embedding idea based on the embedding vector generated for each key point by the network, and finally, the lane line prediction results of the visualization model are visualized by the output module;

[0063] The input image initialization module consists of three convolution layers with a kernel size of 3×3, a stride of 2, and a padding of 1. After each convolution operation, the Prelu activation function and BN normalization are performed. After data preprocessing, the input image enters the initialization module, and the image size will be compressed to a resolution of 32×64, and the image channel dimension will be expanded from 3 to 32;

[0064] The parallel multi-scale feature aggregation network in S3 can integrate the global spatial information of multiple scale feature maps in parallel and enrich the lane feature information; the network architecture of the parallel multi-scale feature aggregation network consists of an encoder and a decoder, which are connected in parallel.

[0065] like Figure 3 As shown in (a), the encoder is based on ResNet-50, which includes a total of 4 stages of parallel subnetworks. It starts with a 32×64 resolution subnetwork as the first stage. With each additional stage, a low-scale parallel subnetwork with half the resolution and double the number of channels is added. Each stage contains 4 residual units, and the last downsampling layer consists of a 3×3 convolution with a stride of 2 and a 3×3 convolution with a stride of 1. In each stage, repeated multi-scale feature fusion is performed between parallel subnetworks, which can effectively extract rich multi-scale feature information. The encoder will output multiple scale feature information through four convolutional layers of different scales at the end.

[0066] The decoder structure is as follows Figure 3 As shown in (b), it consists of 6 dual-head attention feature fusion upsampling modules (DAFU) and 4 feature fusion layers. In this part, the low-to-high multi-scale features generated by the encoder will be gradually integrated into the low-to-high decoding process. Figure 4As shown in the figure, by aggregating global information from two different receptive fields, interpolation-type coarse sampling and transposed convolution-based fine sampling, along the channel dimension through attention feature fusion, it can alleviate the problem caused by scale changes in upsampling and facilitate the network to identify and detect targets under extreme scale changes. DAFU consists of a transposed convolution refinement branch, an interpolation sampling coarse branch, and an attention feature fusion module, which can be expressed as:

[0067]

[0068] Among them, T(·) represents transposed convolution or deconvolution, B(·) represents interpolation upsampling, M(·,·) represents attention feature fusion, δ is the sigmoid function, and x is the input feature map.

[0069] like Figure 3 As shown in (b), DAFU is used as the upsampling module of the decoder to perform parallel upsampling at multiple spatial scales. Assuming that the encoder in the previous stage generates S feature outputs, the decoding process is divided into S-1 stages. represents the jth feature map of the Kth decoding stage. For the convenience of discussion, we take the first stage decoding as an example: this stage has S feature inputs Generate S-1 outputs As the input of the second stage. First, the lowest scale feature After DAFU sampling, output higher-level scale features Not only as the input for the next stage of decoding At the same time, the features of the previous scale are also combined Forming new fusion features Then, the fusion feature After another upsampling, the output It also serves as the input for the next stage of decoding It will be combined with the previous scale feature Forming new fusion features Until output Combination Forming new fusion features End. The above process is repeated in the subsequent stages, and the entire decoding process can be expressed as:

[0070]

[0071]

[0072]

[0073] Where D(·,·) represents the dual-head attention feature fusion upsampling, express The j-1th feature map of the Kth decoding stage after sampling, express and The new j-1th feature map of the Kth decoding stage generated after fusion.

[0074] The parallel multi-scale feature aggregation network integrates a dual-headed attention feature fusion upsampling module (DAFU). DAFU aggregates global information from two different receptive fields, interpolation coarsening sampling and transposed convolutional refinement sampling, along the channel dimension through attention feature fusion, and can accurately restore low-resolution feature maps to pixel-level predictions in parallel at multiple spatial scales; DAFU consists of a transposed convolution refinement branch, an interpolation sampling coarsening branch, and an attention feature fusion module, which can be expressed as:

[0075]

[0076] Among them, T(·) represents transposed convolution or deconvolution, B(·) represents interpolation upsampling, M(·,·) represents attention feature fusion, δ is the sigmoid function, and x is the input feature map.

[0077] The global key point detection module predicts a set of key point probability heat maps and a set of key point position offset vector heat maps based on a parallel multi-scale feature aggregation network, accurately locates the position of lane key points, and ensures the geometric accuracy of the final output lane curve; wherein the probability heat map is used to model the probability space of all target key points and predict the probability of lane line key points for each grid in the image; the offset heat map predicts the position offset vector of the lane line key points;

[0078] The global key point detection module applies the balanced focus loss function and the mean square error loss function to train the network for the probability heat map output and the offset heat map output, respectively. In the training data, each lane curve is marked as a set of ordered key point sets l m ={k i |i=1,2,…,n}, where l m Indicates that the mth lane contains n key points, k i represents the i-th key point; the probability heat map loss and position offset loss are expressed as:

[0079]

[0080]

[0081] Among them, L kp is the key point probability heat map loss, L position is the key point position offset loss, P iis the probability score of the key point at the i-th grid position in the heat map, N k is the total number of key points in the current image, g i represents the true value of positive and negative samples, α is the balance factor, which balances the positive and negative sample losses in the training process, γ is an adjustable hyperparameter, which controls the penalty for negative samples, and P i_x and P i_y are the horizontal and vertical axis offset predictions, G i_x and G i_y is the true value.

[0082] The key point association clustering module further subdivides the global detection key point results of the previous part to determine the lane category to which they belong. For this part, the network generates a feature embedding vector for all detected key points, and trains the network to complete the clustering task by applying an association embedding clustering loss function to the key point feature embedding vector. The key point association clustering loss function of this part can be expressed as:

[0083]

[0084]

[0085]

[0086] L GP =L assemble +L separate

[0087] in, is a keypoint embedding of lane l, K l is the total number of keypoint embeddings for lane l, N represents the total number of lanes in the image, and μ l represents the cluster center of l (average embedding), ||·||2 represents the L2 distance, Δ represents the distance threshold between cluster centers, and L assemble is the ensemble loss, which makes the keypoint embeddings of the same lane category as close as possible to the cluster center μ of the lane embedding. l ; L separate It is the separation loss, which separates the cluster centers of different lane key points embedding so that the key points of different lane categories are far away from each other. The total clustering loss is the sum of the set loss and the separation loss.

[0088] The key point association clustering module uses distance-based clustering technology to segment the key points of each lane line through an iterative process after the model training converges. The specific operations are as follows:

[0089] 1) First, set the embedding feature distance threshold. To avoid selecting an outlier as the threshold, use mean shift to make it closer to the cluster center.

[0090] 2) Select any lane keypoint embedding and assign a lane line label to it. The value of the keypoint embedding is used as the cluster center of the lane line embedding;

[0091] 3) Select an embedding from the remaining key point embeddings and calculate the feature distance (L2 distance) between it and . If the result is less than or equal to the distance threshold, assign the same label to the embedding and take the average of the sum as the cluster center of the new lane line embedding. Otherwise, continue to judge the next point.

[0092] 4) Repeat step 3, traverse all remaining key point embeddings, and find all key point embeddings of lane lines;

[0093] 5) The remaining keypoint embeddings that are not assigned lane labels are iterated again, and steps 2, 3, and 4 are repeated until all keypoint embeddings are assigned a lane label.

[0094] The image output module visually marks the coordinates of the key points after refinement and clustering in the original input image, and distinguishes each lane key point with a different color mark, and outputs the final prediction result.

[0095] S4, feeding the lane line image dataset into the detection model for training, adjusting the model hyperparameters, and training the model until the loss is fully converged to obtain a trained detection model;

[0096] In S4, the image batch size is set to 8, and the model is trained for 300 batches using the Adam optimizer with a learning rate of 0.0001 and the “poly” learning rate decay strategy. The α, γ, Δ, and δ in the loss function are adjusted. i The model loss gradually converges to the minimum value and saves the optimal model.

[0097] S5. Input the image or video to be tested into the trained and converged detection model. The model performs lane line detection and outputs lane line detection information of the image or video.

[0098] In order to verify the implementation example of the present invention, the algorithm proposed in the present invention is calculated and compared with several current mainstream lane line detection algorithms on the TuSimple dataset and the CULane dataset.

[0099] Table 1 gives the specific evaluation results of the Tusimple dataset. Accuracy is the main evaluation indicator of this dataset, which is defined as the average number of correctly predicted lane points in each image, and the calculation formula is as follows:

[0100]

[0101] Among them, C imrepresents the number of points correctly predicted for a given image, S im Represents the number of true points in a given image. When the difference between the true point and the predicted point is less than a certain threshold, the point is predicted correctly.

[0102] Table 1 Comparison results of different algorithms on TuSimple dataset

[0103]

[0104] Table 2 shows the specific evaluation results of the CULane dataset. CULane includes nine different road scenes (such as congestion, shadow, night, glare, etc.). In this dataset, the F1 score is the main evaluation indicator, and the definition formula is:

[0105]

[0106] Among them, the precision rate Indicates the proportion of samples whose predicted value is 1 and whose true value is also 1 among all samples whose predicted value is 1; recall rate It indicates the proportion of samples whose predicted value is 1 and whose true value is also 1 among all samples whose true value is 1.

[0107] Table 2 Evaluation results of different algorithms on the CULane dataset

[0108]

[0109] The results show that the detection accuracy and F1 score of the proposed method are 96.82% and 75.6% respectively, which are better than the current mainstream lane detection method, and the real-time detection rate is 25ms, which has higher robustness and meets the standards of real-time applications.

[0110] Figure 5 The lane line detection visualization result diagram of the model of the present invention is shown.

[0111] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A parallel multi-scale feature aggregation lane detection method based on key point estimation, characterized in that: The method comprises the following steps: S1. Obtain a lane line image dataset for training a lane line detection model; S2, preprocessing the obtained lane line image dataset, uniformly scaling the image size and normalizing it, and applying multiple data enhancement strategies; S3. Construct a parallel multi-scale feature aggregation lane detection model based on key point estimation; The detection model includes an input image initialization module, a parallel multi-scale feature aggregation network, a global key point detection module, a key point association clustering module, and an image output module; after the input image passes through the input image initialization module, it is fed into the parallel multi-scale feature aggregation network for feature extraction, the global key point detection module accurately locates the global lane line key points based on the feature map output by the network, and then the key point association clustering module integrates the global key point detection results into the local lane curve geometry modeling based on the embedding vector generated by the network for each key point, and finally, the lane line prediction results of the visualization model are visualized by the output module; The key point association clustering module in S3 further subdivides the global detection key point results of the previous part to determine the lane category to which they belong. For this part, the network generates a feature embedding vector for all detected key points. By applying an association embedding clustering loss function to the key point feature embedding vector, all key point embeddings are assigned a lane label, and the network is trained to complete the clustering task. The key point association clustering loss function of this part can be expressed as: L GP =L assemble +L separate in, is a keypoint embedding of lane l, K l is the total number of keypoint embeddings for lane l, N represents the total number of lanes in the image, and μ l represents the cluster center of l (average embedding), ||·||2 represents the L2 distance, Δ represents the distance threshold between cluster centers, and L assemble is the ensemble loss, which makes the keypoint embeddings of the same lane category as close as possible to the cluster center μ of the lane embedding. l ; L separate It is the separation loss, which separates the cluster centers of different lane key points embedding so that the key points of different lane categories are far away from each other. The total clustering loss is the sum of the set loss and the separation loss.

2. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The method further comprises the steps of: Feed the lane line image dataset into the detection model for training, adjust the model hyperparameters, and train the model until the loss is fully converged to obtain a trained detection model; The image or video to be tested is input into the trained and converged detection model. The model performs lane line detection and outputs the lane line detection information of the image or video.

3. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The lane line image dataset in S1 includes TuSimple and CULane lane line datasets; The TuSimple dataset is a highway scene at different times of the day under good and moderate weather conditions, including 3626 training images and 2782 test images. Accuracy is the evaluation indicator of the dataset. The CULane dataset has 88,880 training images, 9,675 validation images, and 34,680 test images, covering nine different road scenes, and the F1 score is its evaluation indicator.

4. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The data preprocessing method of the lane line image dataset in S2 is: the image size is uniformly adjusted to a resolution of 256×512, and the image RGB value is normalized to 0-1.

5. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: After data preprocessing of the input image initialization module in S3: the input image size is compressed to 32×64 resolution, and the image channel dimension is also expanded from 3 to 32.

6. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The parallel multi-scale feature aggregation network in S3 can integrate the global spatial information of multiple scale feature maps in parallel and enrich the lane feature information; the network architecture of the parallel multi-scale feature aggregation network consists of an encoder and a decoder, which are connected in parallel.

7. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 6, characterized in that: The parallel multi-scale feature aggregation network in S3 integrates a dual-headed attention feature fusion upsampling module. DAFU aggregates global information from two different receptive domains, interpolation coarsening sampling and transposed convolutional refinement sampling, along the channel dimension through attention feature fusion, and can accurately restore low-resolution feature maps to pixel-level predictions in parallel at multiple spatial scales; DAFU consists of a transposed convolution refinement branch, an interpolation sampling coarsening branch, and an attention feature fusion module, which can be expressed as: Among them, T(·) represents transposed convolution or deconvolution, B(·) represents interpolation upsampling, M(·,·) represents attention feature fusion, δ is the sigmoid function, and x is the input feature map.

8. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The global key point detection module in S3 predicts a set of key point probability heat maps and a set of key point position offset vector heat maps based on a parallel multi-scale feature aggregation network, accurately locates the position of lane key points, and ensures the geometric accuracy of the final output lane curve; wherein the probability heat map is used to model the probability space of all target key points and predict the probability of lane line key points for each grid in the image; the offset heat map predicts the position offset vector of the lane line key points; The global key point detection module applies the balanced focus loss function and the mean square error loss function to train the network for the probability heat map output and the offset heat map output, respectively. In the training data, each lane curve is marked as a set of ordered key point sets l m ={k i |i=1,2,…,n}, where l m Indicates that the mth lane contains n key points, k i represents the i-th key point; the probability heat map loss and position offset loss are expressed as: Among them, L kp is the key point probability heat map loss, L position is the key point position offset loss, P i is the probability score of the key point at the i-th grid position in the heat map, N k is the total number of key points in the current image, g i represents the true value of positive and negative samples, α is the balance factor, which balances the positive and negative sample losses in the training process, γ is an adjustable hyperparameter, which controls the penalty for negative samples, and P i_x and P i_y are the horizontal and vertical axis offset predictions, G i_x and G i_y is the true value.

9. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The key point association clustering module uses distance-based clustering technology to segment the key points of each lane line through an iterative process after the model training converges. The specific operations are as follows: 1) First, set the embedding feature distance threshold. To avoid selecting an outlier as the threshold, use mean shift to make it closer to the cluster center. 2) Select any lane keypoint embedding and assign a lane line label to it. The value of the keypoint embedding is used as the cluster center of the lane line embedding; 3) Select one embedding from the remaining key point embeddings and calculate the L2 distance between it and . If the result is less than or equal to the distance threshold, assign the same label to the embedding and take the average of the sum as the cluster center of the new lane line embedding. Otherwise, continue to judge the next point. 4) Repeat step 3, traverse all remaining key point embeddings, and find all key point embeddings of lane lines; 5) The remaining keypoint embeddings that are not assigned lane labels are iterated again, and steps 2, 3, and 4 are repeated until all keypoint embeddings are assigned a lane label.

10. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: The image output module in S3 visually marks the coordinates of the key points after refinement and clustering in the original input image, and distinguishes each lane key point with a different color mark, and outputs the final prediction result.

11. The method for lane line detection based on parallel multi-scale feature aggregation based on key point estimation according to claim 1, characterized in that: In S4, the image batch size is set to 8, and the model is trained for 300 batches using the Adam optimizer with a learning rate of 0.0001 and the "poly" learning rate decay strategy. The hyperparameters in the loss function are adjusted so that the model loss gradually converges to the minimum value and the optimal model is saved.

Images