Self-adaptive lane detection method based on curvature and edge perception optimization

Abstract

The invention relates to an adaptive lane detection method based on curvature and edge perception optimization, and belongs to the technical field of automatic driving. According to the method, the adaptability of various scenes is enhanced by introducing an adaptive weight mechanism, complex geometric features are captured by fusing Kolmogorov-Arnold Networks (KAN) convolution, and complex lane shapes and edge details are better processed by combining curvature constraint loss and edge detection loss. Through a series of improvements, good balance between the speed and the detection precision is realized, so that the lane line detection system can efficiently and stably operate in an automatic driving environment with limited resources. In order to verify the effectiveness of the proposed method, wide experiments are performed on a CULane data set. Experimental results show that the F1-score of the KAN-Lane in the curve and shielding scenes is 82.08% and 75.44% respectively, the detection precision and stability are remarkably improved, the method is superior to mainstream algorithms such as CLRNet and the like, and the efficient detection speed is kept. The KAN-Lane provides a more reliable lane detection scheme for automatic driving.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and particularly relates to an adaptive lane detection method optimized based on curvature and edge perception. Background Art

[0002] Today, with the rapid development of autonomous driving technology [1], lane line detection, as one of the key technologies of intelligent driving systems, is directly related to the path planning and driving safety of autonomous vehicles. Its main task is to identify the position and shape of lane lines on the road through images captured by a camera in front of the vehicle, so as to assist the vehicle in staying on the correct lane. Especially in complex road scenarios (such as curves, occlusions, night, shadows or strong light conditions), accurately and stably detecting lane lines is crucial for the safe operation of autonomous driving systems [2].

[0003] Existing lane line detection methods can be divided into two categories: traditional image processing-based methods and deep learning-based methods. Traditional image processing-based lane detection methods rely on image processing techniques and geometric models, and usually use algorithms such as edge detection, Hough transform, and curve fitting. Specifically, the combination of the Canny edge detector [3] and the Hough transform [4] is often used to detect straight lane lines, and this method has high computational efficiency when dealing with simple road environments. However, when encountering complex road scenarios (such as curves, multi-lanes or bad weather), the adaptability of traditional methods is poor, and they are easily affected by road marking occlusions, wear or light changes, resulting in low detection accuracy. The robustness of this method in complex scenarios is limited and it is difficult to meet the high requirements of autonomous driving.

[0004] With the progress of Convolutional Neural Network (CNN) in the field of computer vision, deep learning techniques have been widely applied to lane detection. It mainly includes segmentation-based, parameter curve fitting-based, and anchor-based detection methods. The segmentation-based lane detection method regards lane line detection as an image-level semantic segmentation task, and uses structures such as U-Net [6] and Fully Convolutional Neural Network (FCN) [5] to classify each pixel in the image to distinguish lane lines and background regions. The SCNN [7] model introduces a spatial convolution layer to optimize the segmentation effect of lane lines by transmitting information layer by layer. LaneNet [8] uses a fully convolutional network for lane segmentation. The segmentation-based lane detection method performs well in detecting complex lane shapes such as curves or intersections, but it has high computational resource requirements and is not very friendly to real-time applications. The parameter curve-based lane detection method describes the shape of lane lines by fitting polynomial curves or spline curves. PolyLaneNet [9] uses a neural network to predict the polynomial coefficients of lane lines, thereby generating lane line curves. The parameter curve-based lane detection method has strong expressive ability for lane line shapes and is suitable for real-time applications, but its dependence on curve fitting assumptions makes it perform poorly in irregular or complex scenarios. The anchor-based lane line detection method draws on the anchor mechanism in object detection and transforms the lane line detection task into a series of point detection tasks. Line-CNN

[10] predicts the lane line attributes of each anchor by pre-defining the anchor positions in the image and connects them into continuous lane lines in the post-processing step. LaneATT

[11] uses an attention mechanism to enhance the attention to features. The anchor-based lane line detection method has high computational efficiency and is applicable to various complex scenarios, but its detection accuracy is limited when facing sharp-curved lanes.

[0005] To achieve fast and high-precision lane line detection, Chen et al.

[12] implemented the SRLane model through the "Sketch-and-Refine" detection paradigm. This method generates initial lane proposals through rapid estimation of local directions and then optimizes lane line detection through refined adjustment. Although SRLane achieves a good balance between real-time performance and detection accuracy, there is still room for improvement in its detection accuracy when dealing with complex scenarios such as large curvature changes or partial occlusions of lane lines.

[0006] Existing lane line detection methods can usually achieve high recognition accuracy in conventional scenarios, but there are still many technical problems when facing complex environments such as curves, shadows, and strong light. The specific disadvantages are as follows:

[0007] (1) When the vehicle is moving rapidly, it may cause inaccurate autofocus in the corner area captured by the camera;

[0008] (2) In strong light scenarios (such as direct sunlight or reflected light), it is easy to cause overexposure of local images;

[0009] (3) In shadow scenarios (such as under a tree or under a bridge), it may cause underexposure of local areas;

[0010] (4) Occlusions by surrounding buildings, trees or other vehicles will further interfere with the recognition of lane lines.

[0011] These problems increase the difficulty of the model in edge detection and feature extraction, resulting in a decrease in image contrast and making it difficult to distinguish lane lines from the background, seriously affecting the performance of the model.

[0012] At the same time, in actual autonomous driving application scenarios, computing resources are usually limited and the real-time requirement for image processing is relatively high. However, existing deep learning models often have problems such as a large number of parameters and high computational complexity, and it is difficult to run efficiently on resource-constrained platforms. In addition, the complex and dynamic road environment puts forward higher requirements for the full utilization of multi-scale information, but traditional image processing methods and deep learning models often cannot effectively process global information and local details at the same time, resulting in insufficient enhancement effects and limiting the applicability and robustness of the model. Summary of the Invention

[0013] The purpose of the present invention is to solve the problems existing in the background technology, and provide an adaptive lane detection method based on curvature and edge perception optimization (abbreviated as KAN-Lane), which achieves a good balance between speed and detection accuracy, enables the lane line detection system to operate efficiently and stably in a resource-constrained autonomous driving environment, and adapts to the changes in complex lane environments at the same time.

[0014] To achieve the above purpose, the technical solution of the present invention is: an adaptive lane detection method based on curvature and edge perception optimization, including:

[0015] Introduce an improved geometric perception KAN convolution to capture complex geometric features, accurately adapt to the curvature changes of lane lines, and reduce detection errors in corner scenarios;

[0016] Design an improved adaptive weight mechanism to adaptively adjust the weight distribution of multi-level feature fusion according to different scenarios, and be able to focus on key features when facing simple scenarios, reducing resource consumption;

[0017] Propose a multi-scale comprehensive loss function, introduce the training process, constrain the geometric shape and boundaries of lane lines, and ensure that the detected lane lines are smooth and continuous.

[0018] In an embodiment of the present invention, the implementation process of the method is as follows:

[0019] S1. Capture real-time road images through the vehicle's front-facing camera;

[0020] S2. Preprocess the images obtained in step S1;

[0021] S3. Extract image features from the preprocessed images in step S2 through a pre-trained ResNet-18 convolutional neural network;

[0022] S4. In the sketch stage, use the improved geometric perception KAN convolution to obtain a local direction prediction map from the extracted image features and initialize the lane proposals;

[0023] S5. In the refinement stage, use the improved adaptive weight mechanism to perform multi-level feature perception to extract multi-scale fusion features;

[0024] S6. Generate the final lane line prediction results through the classification branch and the regression branch, and perform post-processing including non-maximum suppression to ensure that the finally output lane lines are clear and accurate.

[0025] In an embodiment of the present invention, in the training process of the method, a multi-scale comprehensive loss function is introduced to optimize the network parameters, and the multi-scale comprehensive loss function includes an edge detection loss and a curvature constraint loss.

[0026] In an embodiment of the present invention, the improved geometric perception KAN convolution is specifically as follows:

[0027] In the sketch stage, replace the convolutional layer for local direction estimation with a KAN-based convolutional layer. Each KAN-based convolutional layer includes a KANConv2D convolutional layer, layer normalization LayerNorm, and a PRELU activation function; the convolutional kernel size is 3×3, the stride is 1, and the padding is 1; in the KANConv2D convolutional layer, each element of the convolutional kernel is composed of a non-linear function; formally, each element is defined as:

[0028]

[0029] where, w 1 and w 2 are weights; Spline(x) represents performing spline interpolation on the input x, which is used to fit complex non-linear relationships and provide a smooth and differentiable transformation; σ(·) is the Sigmoid activation function; x·σ(x) combines the smooth characteristics of the Sigmoid activation function and the linear response, which can enhance the expression ability of the model;

[0030] In the KANConv2D convolutional layer, the kernel slides over the image and applies the corresponding elements to the corresponding pixel a kl , and then, the output pixel is calculated as the sum of ; K ∈ R N×M represents the KAN kernel, M represents the matrix representation of the image, and the KANConv2D convolutional layer is defined as follows:

[0031]

[0032] The output of each KAN-based convolutional layer is defined as:

[0033] X l = PReLU(LayerNorm(KANConv2D(X l-1 )))

[0034] where is the output feature of the l-th layer.

[0035] In an embodiment of the present invention, the improved adaptive weight mechanism is as follows:

[0036] The weight distribution during sampling is controlled by an adaptive parameter α, enabling the model to dynamically adjust according to different characteristics of the input image. The improved weight function form is:

[0037]

[0038] where represents the feature value of the feature map at the position (x i , y i ); z i is a learnable parameter that determines the sampling weight of the feature point p i at different scales; s represents the stride corresponding to each feature map; s' represents the possible strides for traversing all feature maps to calculate the normalized weight distribution; N p represents the total number of lane feature points; Proj(·) is the projection function; Exp(·) represents the exponential function, i.e., Exp(x) = e x ; α is an adaptive parameter obtained through model training to control the width of the weight distribution.

[0039] In an embodiment of the present invention, the edge detection loss in the multi-scale comprehensive loss function is as follows:

[0040] The edge detection loss uses the Sobel operator to perform edge detection on the input image, generating the edge response map E(x i , y i ) of the image at the position (xi , y i ), and compare it with the edge response of the predicted image to calculate the difference between the two:

[0041]

[0042] The edge detection loss can help the model better separate the lane lines from the background, reducing false detections or missed detections caused by edge blurring or background interference.

[0043] In an embodiment of the present invention, the curvature constraint loss in the multi-scale comprehensive loss function is as follows:

[0044] The curvature constraint loss is achieved by minimizing the curvature change between discrete points of the lane line. The specific calculation formula is as follows:

[0045]

[0046] where k i represents the curvature of the lane line at the i-th point; curvature is a quantity that describes the degree of curve bending and is approximately calculated through the second derivative of discrete points:

[0047]

[0048] where x i and y i respectively represent the x and y coordinates of the i-th point on the lane. By constraining the curvature change of adjacent points, the curvature constraint loss ensures the smoothness of the detected lane line in shape. Especially in scenarios with large curvature changes, it can effectively reduce unnatural sharp bends.

[0049] In an embodiment of the present invention, the multi-scale comprehensive loss function is expressed as follows:

[0050] L total = α·L original + β·L curvature + γ·L edge

[0051] where L original is the basic loss (i.e., the cross-entropy loss for attention weights

[27] , the focal loss for lane classification

[28] , etc.), L curvature is the curvature constraint loss, L edge is the edge detection loss, and α, β, and γ are the weight coefficients of each loss term, which can be adjusted according to actual needs.

[0052] The present invention also provides an adaptive lane detection system optimized based on curvature and edge perception, which is characterized by comprising a memory, a processor, and computer program instructions stored on the memory and capable of being run by the processor. When the processor runs the computer program instructions, the method steps described above can be implemented.

[0053] The present invention also provides a computer-readable storage medium, on which computer program instructions capable of being run by a processor are stored. When the processor runs the computer program instructions, the method steps described above can be implemented.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] (1) KAN convolution is adopted to improve the feature capture ability in complex scenarios. An improved lane detection model is proposed, which uses KAN convolution to improve the capture ability of complex geometric features. Different from traditional convolutional neural networks, KAN convolution can better handle the curvature changes and occlusion problems of lane lines, enabling the model to accurately identify and locate lane lines in complex scenarios such as curves, shadows, and strong light, thereby improving the robustness and accuracy of detection.

[0056] (2) An adaptive weight mechanism is introduced to achieve efficient adaptation to multiple scenarios. A learnable adaptive weight mechanism is adopted in the multi-scale feature fusion process, enabling the model to dynamically adjust the fusion weights of different-scale features according to the scene complexity, thereby significantly improving the adaptability and detection stability of lane detection in diverse scenarios (such as night, strong light, occlusion, etc.).

[0057] (3) A multi-scale comprehensive loss function is designed to optimize the edge detection and curvature constraint of lane lines. By combining the edge detection loss and the curvature constraint loss, a multi-scale comprehensive loss function is constructed, which helps the model better capture the boundary information and curvature changes of lane lines. The curvature constraint loss ensures the smoothness and consistency of the lane line prediction results, while the edge detection loss improves the recognition ability of lane boundaries. This innovation effectively reduces the errors in detection and ensures the smoothness and accuracy of the lane line detection results. This loss function greatly improves the detection accuracy of the model in complex scenarios such as curved roads and blurred edges.

[0058] (4) Significantly improves the detection performance and computational efficiency. Tests on the public lane detection dataset CULane show that the F1-scores of the present invention in complex scenarios (such as curves and occlusions) reach 82.08% and 75.44% respectively, showing significant improvements compared to mainstream methods (such as CLRNet and SRLane), while maintaining a high detection speed. This indicates that the model significantly improves the detection accuracy in complex scenarios while ensuring real-time performance and computational efficiency, providing a more reliable lane detection solution for the field of autonomous driving. Description of the Drawings

[0059] Figure 1 This is the network model architecture of the method of the present invention.

[0060] Figure 2 This is the comparison of the visualization results between KAN-Lane of the present invention and SRLane; in the figure, (a) is the original input image; (b) is the detection result of the SRLane model; (c) is the detection result of the KAN-Lane model. Detailed Embodiments

[0061] The following combines the drawings to specifically illustrate the technical solutions of the present invention.

[0062] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0063] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "comprise" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0064] The present invention provides an adaptive lane detection method based on curvature and edge perception optimization, including:

[0065] Introduce an improved geometric perception KAN convolution to capture complex geometric features, accurately adapt to the curvature changes of lane lines, and reduce detection errors in curve scenarios;

[0066] Design an improved adaptive weight mechanism to adaptively adjust the weight distribution of multi-level feature fusion according to different scenarios, which can focus on key features in the face of simple scenarios and reduce resource consumption;

[0067] A multi-scale comprehensive loss function is proposed and introduced into the training process to constrain the geometry and boundaries of lane lines, ensuring that the detected lane lines are smooth and continuous.

[0068] As Figure 1 shown, an adaptive lane detection method based on curvature and edge perception optimization in the present invention significantly enhances the model's ability to capture complex geometries by introducing a KAN (Kolmogorov-Arnold Networks)

[13] convolution module, can accurately adapt to the curvature changes of lane lines, reducing the detection error of traditional methods in curved road scenarios; also designs an adaptive weight mechanism to adaptively adjust the weight distribution of multi-level feature fusion according to different scenarios, which can focus on key features in the face of simple scenarios and reduce resource consumption; strengthen the features of key scales in the face of complex scenarios to improve the detection accuracy and robustness; and propose a multi-scale comprehensive loss function, introducing edge detection loss and curvature constraint loss into the training process, effectively constraining the geometry and boundaries of lane lines, ensuring that the detected lane lines are smooth and continuous, and significantly improving the detection stability of the model in complex scenarios. Through this series of improvements, the present invention achieves a good balance between speed and detection accuracy, enabling the lane line detection system to operate efficiently and stably in resource-constrained autonomous driving environments.

[0069] Introduction to the overall process of the method of the present invention:

[0070] The input of the method of the present invention is a real-time road image captured by a vehicle's front camera, which first undergoes a series of data preprocessing, including a series of data augmentation techniques such as image scaling, horizontal flipping, brightness adjustment, blurring, and affine transformation to enhance the generalization ability of the model, and the image is cropped and scaled to a size of 800×320, removing the irrelevant area at the top of the image to match the requirements of the network input tensor dimension. Next, through the pre-trained ResNet-18

[14] The convolutional neural network extracts image features. Then, in the sketch stage, an improved geometric-aware KAN convolution is used to obtain a local direction prediction map from the extracted last-layer feature map, and the lane proposals are initialized through this predicted local direction map. After that, in the refinement stage, an improved adaptive weight mechanism is used for multi-level feature perception to extract multi-scale fusion features. This adaptive weight mechanism can dynamically select the most relevant scale for feature fusion in different scenarios, ultimately enhancing the model's performance in diverse scenarios. Subsequently, the final lane line prediction results are generated through the classification and regression branches, and post-processing steps such as non-maximum suppression are performed to ensure that the finally output lane lines are clear and accurate. During the training process, the network parameters are optimized by combining the introduced multi-scale comprehensive loss function, including edge detection loss and curvature constraint loss. The geometric-aware KAN convolution, dynamic regulation of adaptive weights, and multi-scale comprehensive loss function are introduced in detail in the following modules.

[0071] 1) Geometric-aware KAN Convolution

[0072] To further improve the model's ability to estimate local directions and provide a more flexible and general non-linear function approximation ability, aiming to more effectively approximate complex multi-dimensional functions and better represent local geometric features in complex non-linear spaces, so as to be able to more accurately fit the complex geometric shapes of lanes and ultimately improve the overall performance of lane detection. The present invention introduces KAN convolution, which is a neural network structure based on the Kolmogorov-Arnold theorem and can adapt to scenarios with large curvature changes. The specific implementation method is as follows:

[0073] In the sketch stage, the present invention retains most of the designs in the original framework (i.e., the SRLane

[26] network that completes lane line detection through sketch-refinement operations), and only replaces the convolutional layer for local direction estimation with a KAN-based convolutional layer. Specifically, each KAN-based convolutional module includes a KANConv2D convolutional layer, layer normalization LayerNorm

[15] and PRELU

[16] activation function. The convolutional kernel size is 3×3, the stride is 1, and the padding is 1. In KANConv2D, each element of the convolutional kernel is composed of a non-linear function. Formally, each element is defined as:

[0074]

[0075] where, w 1 and w 2is the weight; Spline(x) represents spline interpolation on the input x, which is used to fit complex non-linear relationships and provide a smooth and differentiable transformation; σ(·) is the Sigmoid activation function; x·σ(x) combines the smooth characteristics of the Sigmoid activation function and the linear response, which can enhance the expressive power of the model.

[0076] In KANConv2D, the kernel slides over the image and applies the corresponding elements to the corresponding pixel a kl , next, the output pixel is calculated as the sum of. K ∈ R N×M represents the KAN kernel, M represents the matrix representation of the image, and KANConv2D is defined as follows:

[0077]

[0078] The output of each convolutional block can be defined as:

[0079] X l = PReLU(LayerNorm(KANConv2D(X l-1 )))

[0080] where belongs to the output features of the l-th layer.

[0081] Here, KANConv2D introduces a complex transformation mechanism based on B-spline, and utilizes its non-linear decomposition ability, enabling local direction prediction to be performed in a higher-dimensional non-linear space. Compared with traditional convolution, the convolution module based on KAN can handle more subtle and complex patterns in the input data, endowing the model with stronger feature capture ability and expressive power, thus enabling it to better capture the direction information of complex curves.

[0082] 2) Dynamic regulation of adaptive weights

[0083] In order to enable the model to selectively focus on information at different scales when processing simple and complex scenarios, and achieve more efficient feature integration and proposal optimization, the present invention introduces an adjustable parameter α into the original weighted summation function based on Gaussian distribution , so that the model can adapt to the diverse feature requirements in complex scenarios. The specific implementation method is as follows:

[0084] The present invention improves the original weight function, and controls the weight distribution during sampling through an adjustable parameter α, enabling the model to dynamically adjust according to different characteristics of the input image. The improved form of the weight function is:

[0085]

[0086] wherein represents the eigenvalue of the feature map at position (x i , y i ); z i is a learnable parameter that determines the sampling weight of the feature point p i at different scales; s represents the stride corresponding to each feature map; s' represents the possible strides for traversing all feature maps to calculate the normalized weight distribution; N p represents the total number of lane feature points; Proj(·) is the projection function; Exp(·) represents the exponential function, i.e., Exp(x) = e x ; α is an adaptive parameter obtained through model training, which is used to control the width of the weight distribution.

[0087] By introducing this parameter, the model can adaptively select the most relevant scale for feature fusion in different scenarios, improving its performance in diverse scenarios; at the same time, it effectively reduces the dependence on non-critical features in some simple scenarios and optimizes the computational efficiency.

[0088] 3) Multi-scale comprehensive loss function

[0089] In order to further improve the robustness and accuracy of the lane detection model and prevent the lane lines from having unnatural sharp bends or being confused with background interference objects in curved roads or complex scenarios, based on the original model, the present invention introduces curvature constraint loss and edge detection loss to better handle complex lane shapes and edge details. The curvature constraint loss aims to ensure the smoothness of the lane lines, while the edge detection loss further utilizes the image edge information to improve the model detection accuracy. The specific definition of the loss function is as follows:

[0090] Curvature constraint loss L curvature : To ensure the smoothness and naturalness of the detected lane lines in terms of geometry, the present invention adds curvature constraint loss. This loss is achieved by minimizing the curvature change between discrete points of the lane lines, and the specific calculation formula is as follows:

[0091]

[0092] where k i represents the curvature of the lane line at the i-th point; curvature is a quantity that describes the degree of curve bending and can be approximately calculated through the second derivative of discrete points:

[0093]

[0094] where x i and y iThey represent the x and y coordinates of the i-th point on the lane respectively. By constraining the curvature change of adjacent points, this loss term ensures the smoothness of the detected lane lines in shape. Especially in scenarios with large curvature changes, it can effectively reduce unnatural sharp bends.

[0095] Edge detection loss L edge : To enhance the model's sensitivity to lane line edge information, the present invention introduces edge detection loss. The Sobel operator is used to perform edge detection on the input image to generate an edge response map E(x i , y i ), and it is compared with the edge response of the predicted image to calculate the difference between the two:

[0096]

[0097] This loss term can help the model better separate lane lines from the background, reduce false detections or missed detections caused by edge blurring or background interference, and improve the overall detection accuracy of the model.

[0098] Combined loss function: The basic loss functions (i.e., the cross-entropy loss for attention weights

[27] , the focal loss for lane classification

[28] , etc.) are represented as L original . On this basis, the curvature constraint loss and the edge detection loss are added to form a new total loss function L total , and the formula is as follows:

[0099] L total = α·L original + β·L curvature + γ·L edge

[0100] Where α, β, and γ are the weight coefficients of each loss term and can be adjusted according to actual needs.

[0101] To evaluate the performance of the lane line detection model in complex scenarios, the present invention uses the public lane line detection dataset CULane

[17] Verify on it. CULane is an open-source large-scale dataset widely used in the lane detection task. This dataset is specifically for the lane detection task in the context of autonomous driving, covering a variety of complex driving scenarios and weather conditions. Its main features are large scale and high diversity, which provide a rich test platform for the development and evaluation of algorithms. The dataset contains more than 130,000 images, divided into three subsets: 88,880 training set images, 9,675 validation set images, and 34,680 test set images. These images are collected by the vehicle's front camera and have a high-definition resolution (1640×590 pixels). They cover a large number of scenes with different road conditions, weather conditions, and driving times, including normal scenes, lane-less scenes, curved lanes, congested scenes, night scenes, shadow scenes, etc. The diversity of these scenes simulates various complex situations in actual driving, thus providing a comprehensive test platform for researchers and engineers to evaluate and improve algorithms.

[0102] The method of the present invention is compared with several recently published algorithms, and the roles of each module are explored on the CULane dataset. The following is a brief introduction to several comparison methods:

[0103] UFLDv2

[18] Is an ultra-fast depth lane detection method that combines a hybrid anchor-driven ordinal classification mechanism. It introduces ordinal classification on the basis of traditional anchor methods and optimizes the detection process by classifying and sorting the anchor positions of lane lines.

[0104] PiNet

[19] Is a lane detection method based on key point estimation and point instance segmentation. It decomposes the lane detection task into the localization and instance segmentation of key points, and uses a deep learning model to identify multiple key points on the lane line and connect them to generate a complete lane line.

[0105] LaneATT

[20] Is an end-to-end lane detection method based on instance segmentation. It uses an instance segmentation model to segment each lane line as an independent instance, avoiding the limitations of traditional lane detection methods that rely on geometric assumptions or post-processing.

[0106] LaneAF

[21] Is a multi-lane detection method based on Affinity Fields. It constructs an affinity field to capture the correlation between lane line pixels and uses a deep learning model to simultaneously predict the geometric shape and instance association information of lane lines.

[0107] SGNet

[22] It is a structure-guided lane detection method that improves detection accuracy by integrating the global geometric structure information of lane lines, extracts image features using a deep learning model, and combines global structure constraints to guide the prediction of lane lines, thereby enhancing the model's robustness to complex shapes and occlusion situations.

[0108] FOLOLane

[23] It is a localized lane detection method starting from low-level key points, emphasizing the refined processing of local features. Through a bottom-up strategy, it detects the key points on lane lines and reconstructs the complete lane lines using the relationships of these key points.

[0109] CondLane

[24] It is a top-down lane detection framework based on conditional convolution. The conditional convolution module dynamically adjusts the parameters of the convolution kernel to adapt to the shapes and feature distributions of different lane lines, thereby enhancing the flexibility and accuracy of detection.

[0110] CLRNet

[25] It is a Cross Layer Refinement Network (CLRNet) for efficient lane detection. By designing a cross-layer optimization mechanism, it fuses the detailed information of low-level features with the semantic information of high-level features to improve the accuracy and robustness of lane line detection.

[0111] SRLane

[26] It is a fast and accurate lane detection method called SRLane (Sketch-and-Refine), which combines anchor-based and key-point-based techniques. It generates preliminary lane line proposals in the fast sketch stage (Sketch), and then optimizes the shape and position of lane lines in the refinement stage (Refine), achieving the combination of real-time performance and high accuracy.

[0112] Among them, CLRNet and SRLane are currently algorithms that achieve state-of-the-art performance.

[0113] Regarding evaluation metrics, the present invention uses F1-score as the main evaluation metric to measure the comprehensive performance of the model in different scenarios. F1-score combines precision and recall, and can better balance the accuracy and coverage of the model, especially suitable for dealing with the imbalance of positive and negative samples in the lane line detection task.

[0114] (1) Precision

[0115] Precision represents the proportion of correctly detected lane lines among the lane lines predicted by the model. The higher the precision, the more accurate the model's detection of lane lines. Its calculation formula is:

[0116]

[0117] Among them, True Positive (TP) represents the number of pixel points of correctly detected lane lines; False Positive (FP) represents the number of non-lane line pixel points misdetected as lane lines.

[0118] (2) Recall

[0119] Recall represents the proportion of correctly detected lane lines among all real lane lines. The higher the recall, the more comprehensively the model can detect lane lines and the fewer missed detections. Its calculation formula is:

[0120]

[0121] Among them, False Negative (FN) refers to the number of pixel points of real lane lines missed by the model.

[0122] (3) F1-score

[0123] F1-score is the harmonic mean of precision and recall, taking both into account, and is a comprehensive performance evaluation indicator. In the case of imbalance between precision and recall, F1-score can provide a comprehensive performance evaluation. The calculation formula of F1-score is:

[0124]

[0125] F1-score is a balance between precision and recall. The higher the value, the better the model performs in terms of both accuracy and coverage. It is a commonly used comprehensive performance evaluation indicator, especially suitable for the CULane dataset with unbalanced class distribution.

[0126] 1. Experimental settings

[0127] The experimental environment of the present invention is configured as follows: Intel Xeon Silver 4214R 2.4GHz CPU, 128GB of memory, NVIDIA GeForce RTX 3090 24GB GPU, python 3.8, Pytorch-gpu 1.13.0, CUDA 11.1. In the training stage, the AdamW optimizer is adopted, the initial learning rate is 6e-4, and the cosine annealing scheduler is used for learning rate decay. The Batch Size is set to 40, and the training is carried out for 31 Epochs in total. The input image size is 800×320. Data augmentation is applied during training, including random flipping, affine transformation, color jitter, and JPEG compression.

[0128] 2. Qualitative comparison

[0129] To qualitatively compare the differences between the improved model of the present invention and the benchmark model, the lane line prediction results of the model of the present invention and the benchmark model were visualized for images in the same scenario, specifically as Figure 2 shown. From left to right, each column is the image of the night, strong light, curve, lane line occlusion, and lane line missing scenarios in sequence. It can be intuitively felt from the figure that compared with the original SRLane, KAN-Lane has stronger performance and stability in complex scenarios such as night, curve, strong light, lane line occlusion, and missing. In such scenarios, the model of the present invention can detect complete lane lines, while the benchmark model can only detect half of the lane line data or even fail to detect lane lines at all.

[0130] 3. Quantitative comparison

[0131] The present invention selects a variety of state-of-the-art (SOTA) lane line detection models in recent years, such as UFLDv2

[18] , PINet

[19] , LaneATT

[20] , LaneAF

[21] , SGNet

[22] , FOLOLane

[23] , CondLane

[24] , CLRNet

[25] , SRLane

[26] Compared with the algorithm of the present invention, the experimental results are shown in Tables 1 and 2. The results show that KAN-Lane achieves an F1@50 metric of 79.84%, and its metrics in multiple scenarios are also better than various existing popular lane detection methods, while being able to have a lower latency. Among them, the F1-score of KAN-Lane in the Shadow and No Line cases is 3.28% and 2.54% higher than that of SRLane respectively, and in the Curve case, the F1-score of KAN-Lane is 3.88% higher than that of the original CLRNet (the current SOTA). The experimental results show that compared with the previous advanced lane detection methods, KAN-Lane shows higher accuracy and reliability in the lane detection task, can better adapt to complex road scenarios, especially curves and shadows, etc., and has better robustness.

[0132] Meanwhile, in order to verify the effectiveness of the module proposed in the present invention, the present invention conducted ablation experiments on the CULane dataset, and the results are shown in Table 3.

[0133] First, the SRLane model was used as the baseline model. Then, experiments were conducted by adding a dynamic regulatory mechanism (DRM) with adaptive weights, a multi-scale integrated loss function (MSI), and KANConv2D. The ablation experiment results are shown in Table 3. It can be seen from Table 3 that adding the dynamic regulatory mechanism with adaptive weights improves the F1@50 value of the model and does not reduce the detection speed, which proves that the dynamic regulatory mechanism with adaptive weights can improve the discriminative ability of the model for lane lines. On this basis, further adding the multi-scale integrated loss function and KANConv2D, the F1@50 value of the model reaches 79.84% and the latency is low, indicating that the module proposed in the present invention can effectively improve the detection performance of the lane line model and maintain a comparable detection speed.

[0134] Table 1 Comparison of the overall performance and efficiency of models on the CULane dataset

[0135]

[0136] Note: The bold font represents the optimal result. The unit of Latency is "ms". The superscript "T" indicates that since there is no source code, the latency was not re-measured under the same conditions, and the data reported in the paper was used.

[0137] Table 2 Performance comparison of models in different scenarios on the CULane dataset

[0138]

[0139] Note: The bold font indicates the optimal result. For all metrics except those in the "Cross" category and "Latency", the higher the value, the better the model performance.

[0140] Table 3 Results of the model ablation experiment on the CULane dataset

[0141]

[0142] Note: The bold font indicates the optimal result.

[0143] The method of the present invention enhances the adaptability of the model in complex scenarios compared with the existing solutions. This is mainly reflected in the following aspects:

[0144] 1. Strong dynamic adaptability: By introducing a learnable adaptive weight mechanism, the feature fusion method can be dynamically adjusted according to the scene complexity, significantly improving the adaptability to complex road scenarios.

[0145] 2. Improved geometric feature capture ability: By introducing the KAN convolution module, the model's perception and fitting ability for complex lane geometries is enhanced, which is superior to traditional convolution operations.

[0146] 3. Comprehensive performance optimization: By designing a multi-scale comprehensive loss function, the comprehensive performance of the model in detection accuracy, robustness, and edge processing is significantly improved.

[0147] 4. Real-time guarantee: While maintaining high computational efficiency, better detection results are achieved, with higher real-time performance compared to existing methods.

[0148] 5. Strong robustness: It has significant advantages in diverse data scenarios (such as night, strong light, etc.), and outperforms existing mainstream algorithms in various scenarios of the CULane dataset.

[0149] The present invention also provides an adaptive lane detection system optimized based on curvature and edge perception, which is characterized by including a memory, a processor, and computer program instructions stored on the memory and executable by the processor. When the processor runs the computer program instructions, the method steps as described above can be implemented.

[0150] The present invention also provides a computer-readable storage medium, on which computer program instructions executable by the processor are stored. When the processor runs the computer program instructions, the method steps as described above can be implemented.

[0151] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0152] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0153] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0155] References:

[0156] [1] Zhang Junhui, Li Qing, and Chen Dapeng. "Research Status and Development Trend of Autonomous Driving Technology." Science Technology and Engineering 20.9 (2020): 3394 - 3403.

[0157] [2] Liu Yu, et al. "Review of Vision-Based Lane Detection Algorithms." Automobile Applied Technology 46.22 (2021): 24 - 27.

[0158] [3]Shou-Ming Hou,Chao-Lan Jia,Ya-Bing Wanga,and Mackenzie Brown.A review of the edge detection technology.Sparklinglight Transactions on Artificial Intelligence and Quantum Computing(STAIQC),1(2):26–37,2021.

[0159] [4]Kai Zhao,Qi Han,Chang-Bin Zhang,Jun Xu,and Ming-Ming Cheng.Deephough transform for semantic line detection.IEEE Transactions on Pattern Analysis and Machine Intelligence,44(9):4793–4806,2021.

[0160] [5]Mendes,Caio César Teodoro,Vincent Frémont,and Denis Fernando Wolf."Exploiting fully convolutional neural networks for fast road detection."2016 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2016.

[0161] [6]Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer International Publishing, 2015.

[0162] [7]Xingang Pan, Jianping Shi, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Spatial as deep: Spatial cnn for traffic scene understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.

[0163] [8]Neven, Davy, et al. "Towards end-to-end lane detection: an instance segmentation approach." 2018 IEEE intelligent vehicles symposium (IV). IEEE, 2018.

[0164] [9]Tabelini, Lucas, et al. "Polylanenet: Lane estimation via deep polynomial regression." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.

[0165]

[10] Li, Xiang, et al. "Line-cnn: End-to-end traffic line detection with line proposal unit." IEEE Transactions on Intelligent Transportation Systems 21.1 (2019): 248-258.

[0166]

[11] Tabelini, Lucas, et al. "Keep your eyes on the lane: Real-time attention-guided lane detection." Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2021.

[0167]

[12] Chen, Chao, et al. "Sketch and Refine: Towards Fast and Accurate Lane Detection." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 2. 2024.

[0168]

[13] Liu Z, Wang Y, Vaidya S, et al. Kan: Kolmogorov-arnold networks[J]. arXiv preprint arXiv:2404.19756, 2024.

[0169]

[14] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.

[0170]

[15] Xu J,Sun X,Zhang Z,et al.Understanding and improving layernormalization[J].Advances in neural information processing systems,2019,32.

[0171]

[16] Thakur R S,Yadav R N,Gupta L.PReLU and edge-aware filter-basedimage denoiser using convolutional neural network[J].IET Image Processing,2020,14(15):3869-3879.

[0172]

[17] Pan X,Shi J,Luo P,et al.Spatial as deep:Spatial CNN for trafficscene understanding[C] / / Proceedings ofthe AAAI Conference on ArtificialIntelligence,2018,32(1).

[0173]

[18] Qin Z,Zhang P,Li X.Ultra-fast deep lane detection with hybridanchor driven ordinal classification[J].IEEE Transactions on Pattern Analysisand Machine Intelligence,2022,46(5):2555-2568.

[0174]

[19] Ko Y,Lee Y,Azam S,et al.Key points estimation and point instancesegmentation approach for lane detection[J].IEEE Transactions on IntelligentTransportation Systems,2021,23(7):8949-8958.

[0175]

[20] Neven D, De Brabandere B, Georgoulis S, et al. Towards end-to-end lane detection: An instance segmentation approach[C] / / 2018 IEEE Intelligent Vehicles Symposium(IV). IEEE, 2018:286-291.

[0176]

[21] Abualsaud H, Liu S, Lu D B, et al. LaneAF: Robust multi-lane detection with affinity fields[J]. IEEE Robotics and Automation Letters, 2021, 6(4):7477-7484.

[0177]

[22] Su J, Chen C, Zhang K, et al. Structure guided lane detection[J]. arXiv preprint arXiv:2105.05403, 2021.

[0178]

[23] Qu Z, Jin H, Zhou Y, et al. Focus on local: Detecting lane marker from bottom up via key point[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021:14122-14130.

[0179]

[24] Liu L, Chen X, Zhu S, et al. Condlanenet: A top-to-down lane detection framework based on conditional convolution[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021:3773-3782.

[0180]

[25] Zheng T, Huang Y, Liu Y, et al. CLRNet: Cross layer refinement network for lane detection[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022:898-907.

[0181]

[26] Chen C, Liu J, Zhou C, et al. Sketch and refine: Towards fast and accurate lane detection[C] / / Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(2):1001-1009.

[0182]

[27] Mao, A., Mohri, M., & Zhong, Y. (2023, July). Cross-entropy loss functions: Theoretical analysis and applications. In International conference on Machine learning(pp. 23803-23828). PMLR.

[0183]

[28] Ross, T.Y., & Dollár, G.K.H.P. (2017, July). Focal loss for dense object detection. In proceedings of the IEEE conference on computer vision and pattern recognition(pp. 2980-2988).

[0184] As described above, it is only the preferred embodiment of the present invention, and it is not intended to limit the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. An adaptive lane detection method based on curvature and edge perception optimization, characterized in that: include: The improved geometry-aware KAN convolution is introduced to capture complex geometric features, accurately adapt to the curvature changes of lane lines, and reduce detection errors in curved scenes; An improved adaptive weight mechanism is designed to adaptively adjust the weight distribution of multi-level feature fusion according to different scenarios, which can focus on key features in simple scenarios and reduce resource consumption; A multi-scale comprehensive loss function is proposed and introduced into the training process to constrain the geometry and boundaries of lane lines to ensure that the detected lane lines are smooth and continuous.

2. The adaptive lane detection method based on curvature and edge perception optimization according to claim 1, characterized in that: The implementation process of the method is as follows: S1, real-time road image captured by the vehicle's forward-facing camera; S2, preprocessing the image obtained in step S1; S3, extracting image features from the image preprocessed in step S2 through a pre-trained ResNet-18 convolutional neural network; S4. In the sketching stage, the improved geometry-aware KAN convolution is used to obtain the local direction prediction map from the extracted image features and initialize the lane proposal; S5. In the refinement stage, an improved adaptive weight mechanism is used to perform multi-level feature perception to extract multi-scale fusion features; S6. Generate the final lane line prediction result through the classification branch and the regression branch, and perform post-processing including non-maximum suppression to ensure that the final output lane line is clear and accurate.

3. The adaptive lane detection method based on curvature and edge perception optimization according to claim 2, characterized in that: The method introduces a multi-scale comprehensive loss function to optimize network parameters during the training process, and the multi-scale comprehensive loss function includes edge detection loss and curvature constraint loss.

4. The adaptive lane detection method based on curvature and edge perception optimization according to any one of claims 1 to 3, characterized in that: The improved geometry-aware KAN convolution is as follows: In the sketching stage, the convolutional layer for local direction estimation is replaced with a KAN-based convolutional layer. Each KAN-based convolutional layer contains a KANConv2D convolutional layer, layer normalization LayerNorm, and PRELU activation function; the convolution kernel size is 3×3, the step size is 1, and the padding is 1; in the KANConv2D convolutional layer, each element of the convolution kernel are composed of nonlinear functions; formally, each element is defined as: Where w1 and w2 are weights; Spline(x) represents spline interpolation of input x, which is used to fit complex nonlinear relationships and provide smooth and differentiable transformations; σ(·) is the Sigmoid activation function; x·σ(x) combines the smoothness of the Sigmoid activation function with the linear response, which can enhance the expressiveness of the model; In the KANConv2D convolutional layer, the kernel slides over the image and converts the corresponding elements Applied to the corresponding pixel a kl , next, the output pixel is calculated as The sum of N×M represents the KAN kernel, M represents the matrix representation of the image, and the KANConv2D convolutional layer is defined as follows: The output of each KAN-based convolutional layer is defined as: X l =PReLU(LayerNorm(KANConv2D(X l-1 ))) in Output features belonging to layer l.

5. The adaptive lane detection method based on curvature and edge perception optimization according to any one of claims 1 to 3, characterized in that: The improved adaptive weight mechanism is as follows: An adaptive parameter α is used to control the weight distribution during sampling, so that the model can be dynamically adjusted according to the different characteristics of the input image. The improved weight function is: in Indicates that the feature map is at position (x i ,y i )’s eigenvalue; z i is a learnable parameter that determines the feature point p i Sampling weights at different scales; s represents the step size corresponding to each feature map; s' represents the calculation of the normalized weight distribution by traversing the possible step sizes of all feature maps; N p represents the total number of lane feature points; Proj(·) is the projection function; Exp(·) represents the exponential function, i.e., Exp(x) = e x ; α is an adaptive parameter obtained through model training and is used to control the width of the weight distribution.

6. The adaptive lane detection method based on curvature and edge perception optimization according to claim 3, characterized in that: The edge detection loss in the multi-scale comprehensive loss function is as follows: Edge detection loss uses the Sobel operator to detect the edge of the input image and generates an image at position (x i ,y i )Edge response graph E(x i ,y i ), and compare it with the edge response of the predicted image Compare and calculate the difference between the two: Edge detection loss can help the model better separate lane lines from the background and reduce false detections or missed detections caused by blurred edges or background interference.

7. The adaptive lane detection method based on curvature and edge perception optimization according to claim 3, characterized in that: The curvature constraint loss in the multi-scale comprehensive loss function is as follows: The curvature constraint loss is achieved by minimizing the curvature change between discrete points of the lane line. The specific calculation formula is as follows: Among them, k i Represents the curvature of the lane line at the i-th point; curvature is a quantity that describes the degree of curvature of a curve and is approximately calculated by the second-order derivative of the discrete point: where x i and i They represent the x and y coordinates of the ith point on the lane, respectively. By constraining the curvature changes of adjacent points, the curvature constraint loss ensures the smoothness of the detected lane line in shape, especially in scenes with large curvature changes, which can effectively reduce unnatural sharp bends.

8. The adaptive lane detection method based on curvature and edge perception optimization according to any one of claims 1 to 7, characterized in that: The multi-scale comprehensive loss function is expressed as follows: L total =α·L original +β·L curvature +γ·L edge Among them, L original is the basic loss, L curvature is the curvature constraint loss, L edge is the edge detection loss, α, β and γ are the weight coefficients of each loss term, which can be adjusted according to actual needs.

9. An adaptive lane detection system based on curvature and edge perception optimization, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 8 can be implemented.

10. A computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 8 can be implemented.

Images