Lane line detection method based on lane line detection model and training method of lane line detection model

Abstract

A lane line detection method based on a lane line detection model, a training method of the lane line detection model, an electronic device and a storage medium, relate to the technical field of computer vision and deep learning. The lane line detection model comprises a feature extraction network and a prediction module, and the lane line detection method comprises: extracting image features of a to-be-processed image through the feature extraction network; determining an initial implicit function of a lane line in the to-be-processed image through the prediction module based on the image features, and generating an elastic lane graph based on the initial implicit function; and determining a predicted lane line in the to-be-processed image according to a zero horizontal contour line of the elastic lane graph. The training method of the lane line detection model is: training a neural network through an elastic interaction energy loss function, simplifying gradient calculation of the loss function by using fast Fourier transform, so as to obtain an accurate representation of a target lane line by using parameters of an updated elastic lane graph. The lane line detection model can accurately identify lanes with complex geometric structures and shapes.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the fields of computer vision and deep learning technology, and in particular to a lane line detection method based on a lane line detection model, a training method for the lane line detection model, an electronic device, and a storage medium. Background Technology

[0002] Artificial intelligence is currently widely used in ADAS (Advanced Driving Assistance Systems). Lane detection is a key technology in ADAS, helping intelligent vehicles plan driving maneuvers.

[0003] Lane detection involves real-time identification of the boundaries of the driving area. Besides the real-time requirement, the challenges of lane detection can be divided into two categories: lane scenarios with weak features and lane scenarios with complex geometry. For example... Figure 1a As shown, weak lane feature scenarios refer to situations where lane lines are not clearly visible, typically occurring in the following circumstances: the road is unmarked, obstructed by vehicles and pedestrians, or due to insufficient lighting (such as low visibility caused by shadows, nighttime, or glare). Figure 1b As shown, complex geometry lane scenarios include lanes with complex structures or shapes, such as intersections, large-angle Y-shaped lanes and merging lanes, large-curvature turns, and dense lanes.

[0004] Traditional lane detection methods typically rely on manually designed operators, utilizing image gradients to detect lane edges, such as the Hough transform, followed by post-processing. However, these methods perform poorly when faced with a wide range of perception tasks in various real-world scenarios. Thanks to the rapid development of deep learning, many deep learning-based lane detection methods have demonstrated groundbreaking results in achieving end-to-end understanding of autonomous driving scenarios. However, most methods are designed for straight lanes without branching or lanes with low curvature, while real-world driving scenarios are diverse, with varying lane structures, such as branching lanes at intersections, leading to a lack of robustness when these methods are applied to real-world scenarios.

[0005] For lanes with weak features and complex geometries, it is necessary to explore and develop more robust lane detection models and methods. Summary of the Invention

[0006] Embodiments of this disclosure propose a lane line detection method based on a lane line detection model, a training method for the lane line detection model, an electronic device, and a storage medium.

[0007] In a first aspect, embodiments of this disclosure provide a lane line detection method based on a lane line detection model, the lane line detection model including a feature extraction network and a prediction module. The method includes: extracting image features of the image to be processed through the feature extraction network; determining an initial implicit function of the actual lane lines in the image to be processed through the prediction module based on the image features, and generating an elastic lane map based on the initial implicit function; and determining the predicted lane lines in the image to be processed according to the zero horizontal contour line of the elastic lane map.

[0008] Secondly, embodiments of this disclosure provide a training method for a lane detection model, which includes a feature extraction network and a prediction module. The method includes: extracting image features from a sample image through the feature extraction network; determining an initial latent function of the actual lane lines in the sample image based on the image features through the prediction module, and generating an elastic lane map based on the initial latent function; determining an elastic interaction energy loss function based on the latent functions of the elastic lane map and the real lane line labels in the sample image, and updating the parameters of the lane detection model based on the elastic interaction energy loss function to obtain an updated lane detection model.

[0009] Thirdly, embodiments of this disclosure provide a lane detection method based on a lane detection model, the lane detection model including a feature extraction network and a prediction module. The method includes: extracting image features of the image to be processed through the feature extraction network; and determining predicted lane lines in the image to be processed through the prediction module based on the image features, wherein the predicted lane lines are represented as the derivative of the initial implicit function of the lane lines in the image to be processed.

[0010] Fourthly, embodiments of this disclosure provide a training method for a lane detection model, which includes a feature extraction network and a prediction module. The method includes: extracting image features from a sample image through the feature extraction network; determining predicted lane lines in the image to be processed through the prediction module based on the image features, wherein the predicted lane lines are represented as the derivatives of the implicit functions of lane lines in the image to be processed; determining an elastic interaction energy loss function based on the predicted lane lines and the true lane line labels of the sample image; and updating the parameters of the lane detection model based on the elastic interaction energy loss function to obtain an updated lane detection model.

[0011] Fifthly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement, when executed, a lane detection method based on a lane detection model as described in the first or third aspect, or a training method for a lane detection model as described in the second or fourth aspect.

[0012] In a sixth aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer, when executed, to implement a lane detection method based on a lane detection model as described in the first or third aspect, or a training method for a lane detection model as described in the second or fourth aspect.

[0013] The lane detection method and training method based on the lane detection model provided in the embodiments of this disclosure extract image features from the image to be processed, generate an elastic lane map based on the initial implicit function of the actual lane lines, and use the elastic lane map to implicitly represent the predicted lane lines. The predicted lane lines are constructed as zero-width horizontal contour lines in the elastic lane map. The elastic lane map determined based on the initial implicit function can be applied to curves representing complex geometric structures / shapes and has flexible adaptability to changes in geometric structures / shapes, thereby making the predicted lane line contours closer to the actual lane line contours.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1a and Figure 1b The diagrams show some weak lane feature scenarios and complex geometry lane scenarios in the lane detection task. Figure 2a , Figure 2b , Figure 2c and Figure 2d Schematic diagrams of several major lane detection methods based on deep neural network models are shown respectively; Figure 3 This is an exemplary system architecture diagram in which the lane line detection method or the training method of the lane line detection model based on the lane line detection model of the present disclosure can be applied; Figure 4 This is a flowchart of a lane line detection method based on a lane line detection model provided according to embodiments of the present disclosure; Figure 5 This is a flowchart of a training method for a lane detection model provided according to embodiments of the present disclosure; Figure 6 This is an exemplary block diagram of a lane detection model provided according to embodiments of the present disclosure; Figure 7 This is a schematic diagram of the architecture of the lane detection network ElasticLaneNet provided according to embodiments of the present disclosure; Figure 8a This is a flowchart of a lane line detection method based on a lane line detection model provided according to another embodiment of the present disclosure; Figure 8b This is a schematic diagram of a flexible lane diagram according to an embodiment of the present disclosure; Figure 8c This is a schematic diagram illustrating the process of obtaining specific lane coordinates by line-by-line sampling based on an elastic lane map according to an embodiment of this disclosure; Figure 9 This is a flowchart of line-by-line sampling based on an elastic lane map, provided according to embodiments of the present disclosure; Figure 10 This is a flowchart of a training method for a lane detection model according to another embodiment of the present disclosure; Figure 11 This is a schematic diagram illustrating how a predicted lane line is driven to move in a direction approaching the actual lane line label, guided by an elastic interaction energy loss function, according to an embodiment of this disclosure. Figure 12 This is a flowchart illustrating the auxiliary training of a lane detection model by constructing an auxiliary loss function, according to embodiments of this disclosure. Figure 13 This is a schematic diagram comparing the prediction results of the lane detection network ElasticLaneNet based on this disclosure with other lane detection models on the SDLane dataset; Figure 14 This is a schematic diagram illustrating the prediction results of the lane detection network ElasticLaneNet based on this disclosure in some challenging scenarios on the CULane dataset and the TuSimple dataset; Figure 15 This is a schematic diagram comparing the prediction results of the lane line detection network ElasticLaneNet based on this disclosure in some challenging scenarios on the CULane dataset and the TuSimple dataset with the true lane line labels. Figure 16 This is a variant of the lane detection model of this disclosure, ElasticLaneNet, used for ablation experiments. pw Architecture diagram; Figure 17 This is a schematic diagram of the structure of an electronic device suitable for implementing the lane detection model-based lane detection method or lane detection model training method of the embodiments of this disclosure. Detailed Implementation

[0016] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0017] It should be noted that the collection, updating, analysis, use, transmission, and storage of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations, are used for legitimate and reasonable purposes, are not shared, disclosed, or sold outside of these legitimate uses, and are subject to supervision and management by national regulatory authorities.

[0018] Currently, lane detection methods based on deep neural network models can be mainly classified into the following categories: segmentation-based methods, parameter-based methods, anchor-based methods, and row-wise methods.

[0019] 1. Segmentation-based methods

[0020] refer to Figure 2a As shown, segmentation-based methods require instance-level segmentation. For example... Figure 1a and Figure 1b Different colors (such as blue, green, red, and yellow) can represent different instances. After obtaining pixel-level discrimination results, post-processing steps are needed to determine the position coordinates of each lane. However, treating lanes as dense pixel masks is inefficient, and overlapping lane sections are difficult to separate accurately. Furthermore, when lane features are weak, the predicted lane lines often appear broken or missing.

[0021] 2. Parameter-based methods

[0022] Parametric methods use explicit parameterized functions, such as polynomials, Bézier curves, or explicit functions based on mathematical models, to characterize lanes. For example, one could use... Figure 2b The quadratic polynomial shown ,in,( ) is the first i The coordinates of each sampling point i It is a positive integer. a , b , cThese are polynomial coefficients. However, complex lane shapes are often difficult to represent flexibly using explicit functions. Furthermore, parameter-based methods are highly sensitive to parameter prediction errors; small errors in higher-order coefficients can lead to significant deviations in predicted lane shapes. Therefore, when lane shapes are complex and varied, parameter-based methods struggle to achieve high-precision predictions.

[0023] 3. Anchor-based method

[0024] refer to Figure 2c As shown, anchor-based methods first predefine ROIs (Regions of Interest) as line anchors, select the set of line anchors that are closest to the target lane line at a specified metric level, construct the lane by predicting the offsets of these anchors, and then use traversal NMS (Non-Maximum Suppression) to determine the lane line with the highest confidence as the final prediction result. However, anchor-based methods rely on accurate prior anchor predictions and cannot adapt to diverse structures and shapes, such as intersections, turns, Y-shaped intersections, or dense lanes.

[0025] 4. Row prediction-based methods

[0026] refer to Figure 2d As shown, most row prediction-based methods are based on Coarse Grid Maps (CGMs), which are similar to instance segmentation methods but more efficient. Lane positions (e.g., x-coordinates) in the CGM are output via row expectation values ​​or grid confidence scores. Additional outputs, such as offsets, can be integrated into the CGM through post-processing steps to improve lateral accuracy. Some studies have proposed hybrid sampling directions to address the amplified local errors that easily occur in row prediction-based methods. However, real-world driving scenarios involve various lane structures, and the portions perpendicular to the sampling direction, including intersections, large-angle Y-shaped lanes, and high-curvature curves, are difficult to localize within the CGM.

[0027] Among the lane detection methods based on deep neural network models mentioned above, parameter-based and anchor-based methods are often unable to adapt to lane prediction with complex shapes, while segmentation-based and row prediction-based methods are mostly local processing methods and are prone to classification imbalance problems in multi-instance classification (pixel / grid), such as missing, interrupted or blurred lane lines. Finally, they still need to rely on costly post-processing steps to determine the lane position.

[0028] The present disclosure will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0029] This disclosure provides a lane detection method based on a lane detection model and a training method for the lane detection model. It utilizes a lane detection model based on a deep neural network to generate an Elastic Lane Map (ELM) for implicitly representing lane lines. Compared to parameter-based and anchor-based methods, this method exhibits stronger geometric adaptability for lane recognition with weak lane features and complex geometries, and can better fit the real lane line contours. Guided by the Elastic Interaction Energy (EIE) loss function, the predicted lane lines obtained from the Elastic Lane Map are smooth and coherent curves, and can integrate global information with low-level features. This overcomes the challenges of weak lane features and class imbalance (e.g., missing lanes, interruptions, or blurring) that are prone to occur in segmentation-based methods. Furthermore, compared to segmentation-based and row prediction-based methods, it does not rely on costly post-processing steps to determine lane positions.

[0030] Figure 3 An exemplary system architecture 300 is shown, in which lane detection methods or training methods for lane detection models based on lane detection models can be applied to embodiments of the present disclosure.

[0031] like Figure 3 As shown, system architecture 300 may include terminal device 301, network 302, and server 303. Network 302 is used as a medium to provide a communication link between terminal 301 and server 303. Network 302 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0032] Terminal device 301 interacts with server 303 via network 302 to receive or send messages, etc. Various client applications can be installed on terminal device 301 to interact with server 303.

[0033] Server 303 can be a server that provides various services, such as a backend server that receives requests sent by terminal device 301 with which it has established a communication connection. The backend server can receive and analyze the requests sent by the terminal device, and generate processing results to feed back to the terminal device.

[0034] Terminal device 301 may, for example, capture an image to be processed including road lane markings and send the image to be processed to server 303; server 303 may, for example, process the image to be processed using a lane detection model to obtain a prediction result, which may include at least one predicted lane line, and server 303 may send the prediction result to terminal device 303.

[0035] Server 303 can be either hardware or software. When server 303 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 303 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0036] Server 303 can provide various services through its built-in applications. For example, it can provide lane detection services based on a lane detection model. Server 303 can achieve the following: First, it extracts image features from the image to be processed using a feature extraction network. Then, based on the image features, a prediction module determines the initial hidden function of the lane lines in the image to be processed and generates an elastic lane map based on the initial hidden function. Finally, it determines the predicted lane lines in the image to be processed based on the zero-level contour line of the elastic lane map. The lane detection model can include a feature extraction network and a prediction module. Alternatively, it can provide training services for a lane detection model. Server 303 can achieve the following: First, it extracts image features from sample images using a feature extraction network. Then, based on the image features, a prediction module determines the initial hidden function of the lane lines in the sample images and generates an elastic lane map based on the initial hidden function. Next, it determines the elastic interaction energy loss function based on the hidden functions of the elastic lane map and the true lane label of the sample images, and updates the parameters of the lane detection model based on the elastic interaction energy loss function to obtain the updated lane detection model.

[0037] It should be noted that the method provided in this embodiment can be executed by server 303 or by terminal device 301, and this disclosure does not limit it.

[0038] It should be understood that Figure 3 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0039] Continue to refer to Figure 4 , Figure 4 A flowchart 400 of a lane line detection method based on a lane line detection model according to an embodiment of the present disclosure is shown. In this embodiment, the lane line detection model may include a feature extraction network and a prediction module, and the flowchart 400 may include the following steps: Step 401: Extract image features of the image to be processed using a feature extraction network.

[0040] This step aims to be implemented by the entity executing the lane detection method based on the lane detection model (e.g., Figure 3The server 303 shown extracts image features of the image to be processed through the feature extraction network of the lane line detection model.

[0041] In embodiments of this disclosure, the image to be processed can be a captured road image. The image to be processed may include actual lane markings used to identify road boundaries. The image to be processed may also include other road markings, such as turn arrows, speed reduction signs, etc.

[0042] In an exemplary embodiment, the aforementioned executing entity can input the image to be processed into a feature extraction network. The feature extraction network can extract image features from the image by performing operations such as feature encoding, feature extraction, and feature fusion. For example, the feature extraction network can use a CNN (Convolutional Neural Network) to extract features such as pixel values, edges, shapes, colors, and textures from the image. It can also perform feature fusion and other processing on these features to obtain image features.

[0043] Step 402: Based on image features, the initial latent function of lane lines in the image to be processed is determined by the prediction module, and an elastic lane map is generated based on the initial latent function.

[0044] Based on step 401, this step aims to have the aforementioned executing entity determine the initial latent function of the lane lines in the image to be processed by the prediction module of the lane line detection model based on the extracted image features, and generate an elastic lane map based on the initial latent function.

[0045] In an exemplary implementation, the aforementioned execution entity may generate an elastic lane map based on an initial implicit function, which can be used to implicitly represent the contour of the predicted lane lines.

[0046] In an exemplary implementation, the flexible lane map can be understood as an implicit function representing a curve, with the predicted lane lines modeled as "..." in the flexible lane map. Open curves with no width (Open Curves without Width), these curves are implicitly embedded in the elastic lane map.

[0047] In an exemplary implementation, the predicted lane line profile can be represented as a set of function values ​​for a specific independent variable.

[0048] As an exemplary implementation, the level set function used to construct the initial implicit function is the distance from each pixel in the image to the target curve, such as an actual lane line. When a pixel is to the left of the target curve, the value of the level set function is negative; when a pixel is to the right of the target curve, the value of the level set function is positive; and the target curve corresponds to the position where the value of the level set function is 0.

[0049] Step 403: Determine the predicted lane lines in the image to be processed based on the zero-level contour line of the elastic lane map.

[0050] In this step, the execution entity, based on the elastic lane map generated in step 402, determines the zero horizontal contour line of the elastic lane map and then obtains the predicted lane lines in the image to be processed.

[0051] In an exemplary implementation, as described in step 402, the elastic lane map is based on a level set function, and the predicted lane line is represented as the zero-level contour of the elastic lane map, that is, the position of the curve where the predicted lane line is located is the zero level set where the implicit function has a value of zero.

[0052] The lane detection method based on the lane detection model disclosed in the above embodiments extracts image features from the image to be processed and generates an elastic lane map based on the initial implicit function of the actual lane lines. The elastic lane map is used to implicitly represent the predicted lane lines, and the predicted lane lines are constructed as zero-width horizontal contour lines in the elastic lane map. The elastic lane map determined based on the implicit function can be applied to curves representing complex geometric structures / shapes and has dynamic adaptability to changes in geometric structures / shapes, thereby making the predicted lane line contours closer to the actual lane line contours.

[0053] Next reference Figure 5 , Figure 5 A flowchart 500 illustrating a training method for a lane detection model according to an embodiment of the present disclosure is shown. In this embodiment, the lane detection model may include a feature extraction network and a prediction module, and the flowchart 500 may include the following steps: Step 501: Extract image features from the sample image using a feature extraction network.

[0054] In this step, the aforementioned executing entity (e.g.) Figure 3 The server 303 in the model can perform feature encoding, feature extraction, and feature fusion on the sample images in the training dataset through the feature extraction network of the lane detection model in order to extract the image features of the sample images.

[0055] In an exemplary implementation, the lane detection model's parameters are updated using a training dataset. The training dataset may include multiple sample images, from which pixel values, edges, shapes, colors, textures, and other information can be extracted as features. Each sample image may be labeled, and the label may correspond to the model's expected output. The label, for example, may be the ground truth (GT) of the actual lane lines.

[0056] It should be understood that the features and labels listed above are merely illustrative examples of the sample images and do not constitute a limitation of this disclosure.

[0057] Step 502: Based on image features, the initial latent function of lane lines in the sample image is determined by the prediction module, and an elastic lane map is generated based on the initial latent function.

[0058] In this step, the execution entity can determine the initial implicit function of the lane lines in the sample image through the prediction module based on the image features of the sample image extracted in step 501, and generate an elastic lane map based on the initial implicit function.

[0059] Step 503: Determine the elastic interaction energy loss function based on the implicit function of the elastic lane map and the real lane line labels.

[0060] In this step, the aforementioned executing entity can derive the implicit function of the actual lane lines based on the real lane line labels, and then construct the Elastic Interaction Energy (EIE) loss function based on the implicit function of the actual lane lines and the elastic lane map. The implicit function of the actual lane lines can be determined based on the level set function of the real lane line labels.

[0061] Step 504: Update the parameters of the lane detection model based on the elastic interaction energy loss function to obtain the updated lane detection model.

[0062] In this step, the aforementioned execution entity can update the parameters of the lane detection model based on the constructed EIE loss function to obtain the updated lane detection model. The elastic lane map generated by the updated lane detection model can provide a more accurate representation of the predicted lane position.

[0063] In the embodiments of this disclosure, the EIE loss function based on elastic interaction energy is inspired by the physical properties of linear defects in crystals. Inspired by this elastic interaction energy, for lane detection tasks, the predicted lane line and the actual lane line trajectory (such as the actual lane line label) are likened to two defect lines in a crystal dislocation. An EIE loss function is constructed based on the elastic interaction energy between the two, and this function measures the difference between the predicted lane line and the actual lane line trajectory. During training, guided by the EIE loss function, the predicted lane line is driven to move towards a trajectory closer to the actual lane line trajectory.

[0064] The present disclosure discloses a training method for the lane detection model described above. An EIE loss function is constructed based on the prediction results of the lane detection model and the actual lane trajectory to update the parameters of the lane detection model. During training, guided by the EIE loss function, the predicted lane line is driven to move by minimizing the elastic interaction energy between the two curves, so that it coincides with or approaches the actual lane line, thereby making the predicted lane line contour closer to the actual lane line contour.

[0065] refer to Figure 6 , Figure 6 An embodiment of the present disclosure provides a lane line detection model 600 that can be applied to various servers or electronic devices.

[0066] like Figure 6 As shown, the lane detection model 600 provided in this embodiment may include a feature extraction network 601 and a prediction module 602. The feature extraction network 601 can extract image features from the input image; the prediction module 602 can generate an elastic lane map for implicitly representing lane lines based on the image features extracted by the feature extraction network 601.

[0067] In an exemplary embodiment, the feature extraction network 601 may include a feature extraction subnetwork 611 and a multi-scale feature fusion subnetwork 612. The feature extraction subnetwork 611 may, for example, include a CNN network, which performs convolutions and downsampling through multiple convolutional layers to extract a set of features from an image.

[0068] In an exemplary embodiment, the feature extraction subnetwork 611 may include a residual module, which extracts a set of features from the input image and directly obtains a first feature based on the extracted set of features.

[0069] In some optional implementations, the feature extraction subnetwork 611 may further include a feature enhancement bottleneck layer, which extracts initial features from the input image through the residual module, and performs global feature association extraction and feature enhancement on the initial features through the feature enhancement bottleneck layer to obtain the first feature.

[0070] In some exemplary embodiments, the multi-scale feature fusion subnetwork 612 may include, for example, a Feature Pyramid Network (FPN) module. Based on the first feature output by the feature extraction subnetwork 611, the FPN module performs upsampling and feature fusion on different feature layers to obtain fused image features.

[0071] It should be understood that the above feature extraction submodule and feature fusion submodule are merely illustrative examples of the feature extraction network and do not constitute a limitation of this disclosure.

[0072] In an exemplary embodiment, the prediction module 602 determines an initial implicit function for lane lines in the image based on image features, and generates an elastic lane map based on the initial implicit function. This elastic lane map can model and represent lanes with complex structures or geometries. By modeling lane lines as the zero-level contour lines of the elastic lane map, predicted lane lines in the image can be determined based on the elastic lane map generated by the prediction module.

[0073] The lane detection model 600 in this embodiment can be used to implement... Figure 4 , Figure 5 For steps 401-402 or 501-502 in the corresponding embodiments, the specific processing flow of the relevant steps and the technical effects they bring can be referred to the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0074] The lane detection model of the above embodiments of this disclosure provides a new perspective on lane detection by modeling the predicted lane line as a zero-level contour line in an elastic lane map. The generated elastic lane map has high geometric adaptability and can identify lane lines with complex and variable shapes. The predicted lane line obtained based on the elastic lane map is smooth and coherent, and can fit the trajectory of the actual lane line well even when the sampling points are sparse.

[0075] Continue to refer to Figure 7 , Figure 7 A schematic diagram of the architecture of a lane detection network according to an embodiment of the present disclosure is shown. This embodiment provides an end-to-end lane detection network called ElasticLaneNet.

[0076] Combination Figure 7 As shown, the network architecture of ElasticLaneNet provided in this embodiment may include the following key components: Backbone and ELMM (Elastic Lane Map Module).

[0077] The backbone of ElasticLaneNet acts as a feature extraction network, responsible for extracting features from the input image for subsequent processing and analysis. The backbone can be used to extract global and local features of an image, such as edges, textures, and shapes. It may include multiple convolutional layers or other feature extraction layers to extract features at different scales or levels from the image layer by layer. For example, the output of the backbone network can be a multi-scale feature map.

[0078] ELMM, also known as the prediction module, is used to generate an ELM based on image features extracted from the backbone network. The prediction module may include multiple fully connected layers or convolutional layers to map image features to the final output space, generating the model's final prediction result.

[0079] In this embodiment, the Backbone may further include a residual module and an FPN module. The residual module may use a ResNet (Residual Network) 34 as an encoder to extract a set of features from the image and output this set of features as the first feature to the FPN module; the FPN module is used to perform feature fusion based on the first feature to obtain fused image features.

[0080] In some exemplary implementations, to improve performance, the backbone network also includes a bottleneck layer located between the residual module and the FPN module. The bottleneck layer can perform global information association extraction, feature enhancement, and other processing. Specifically, the bottleneck layer can utilize the self-attention mechanism of the Transformer to capture more global contextual information. In this paper, the bottleneck layer with the Transformer is referred to as the Transformer Bottleneck (feature enhancement, abbreviated as TB) bottleneck layer.

[0081] As an exemplary implementation, the residual module can be used to extract a set of initial features from the image; the feature enhancement bottleneck layer is used to perform global feature association extraction and feature enhancement on the initial features extracted by the residual module to obtain the first feature; the FPN module is used to generate features at different levels based on the first feature, and to fuse all or some of the features at different levels to obtain fused image features, called fused features. The features at different levels may include, for example, high-level features and low-level features. High-level features typically include more global contextual information, while low-level features include more local detail information.

[0082] In some exemplary embodiments, the lane detection network may further include an AFR (Auxiliary Feature Refinement) submodule. The AFR submodule can perform cross-layer feature fusion on features from different levels; for example, it can collect features from one or more levels of features, such as lower-level features, to obtain auxiliary fused features. In an exemplary embodiment, the prediction module can optimize implicit lane prediction based on the auxiliary fused features obtained from the AFR submodule.

[0083] In an exemplary implementation, the auxiliary fusion features obtained in the AFR submodule can be used to assist in training the lane detection model so that the model can more accurately identify lane lines.

[0084] In some exemplary embodiments, the lane detection network may further include a lane presence sub-network (also referred to as a CSN sub-module). The CSN sub-module can be used to predict the existence of lane lines based on a first feature, that is, the predicted probability of the existence of each lane line in the image.

[0085] In some exemplary embodiments, the lane line detection network may further include a lane range sub-network, also known as an RSN sub-module. The RSN sub-module can be used to predict the existence and location of a lane line in a given row based on a first feature, that is, the predicted probability of the lane line existing in each y-coordinate row of the image.

[0086] The embodiments disclosed above provide an end-to-end lane detection network, ElasticLaneNet, including a backbone network and a prediction module. The backbone network can adopt an Encoder-Transformer-FPN structure, and the prediction module outputs an ELM (Elastic Lane Detection Model). The ELM output by ElasticLaneNet provides a novel lane representation method, capable of implicitly representing lane lines with complex geometries and shapes, offering a new perspective on lane modeling. Furthermore, the ELM exhibits high geometric adaptability, capable of recognizing lane lines with complex and variable shapes. Moreover, by comprehensively acquiring global spatial structural relationships and local detailed information, as well as fusing high- and low-level features, even in cases of lane line discontinuity such as occlusion or missing lines, the continuity of the predicted lane lines and the inclusion of complete lane information can be ensured, thereby improving the accuracy of lane line detection.

[0087] To further elaborate on the implementation details of the lane detection network ElasticLaneNet and the lane detection method of the embodiments of this disclosure, Figure 8a A processing flow 800 of a lane line detection method based on a lane line detection model according to another embodiment of the present disclosure is shown.

[0088] refer to Figure 7 and Figure 8a As shown, in this embodiment, process 800 may include the following steps: Step 801: Input the image into the lane detection model.

[0089] In this step, the given image is input into ElasticLaneNet. The given image has a size of H×W, where H is the height and W is the width.

[0090] Step 802: Extract a set of features from the image using the residual module.

[0091] In this step, for the input H×W low-resolution image I∈R H×W×3 By using multiple convolutional layers and downsampling operations, a downsampled image is obtained. I'∈R h×w×3 .

[0092] Taking the ResNet 34 network as an example, the extracted set of features includes the outputs of multiple convolutional layers. C n Feature maps of different resolutions are generated at different stages (feature extraction stages), n=2, 3, 4. The feature map C for each stage... n The dimensions are 1 / 2, 1 / 4 and 1 / 8 of the original image, respectively, meaning that the size of the feature maps of adjacent stages is twice that of the original image.

[0093] Step 803: Perform global feature enhancement on a set of features through the bottleneck layer to obtain the first feature.

[0094] In this step, the extracted set of features can be further processed using the designed Transformer Bottleneck structure. For example, self-attention mechanisms can be used for global feature association extraction and feature enhancement, thereby effectively improving the model's feature representation ability. Furthermore, the bottleneck layer can reduce the number of model parameters and improve the network's generalization ability. In some implementations, the bottleneck layer can employ the Transformer's self-attention mechanism to perform global feature extraction on feature maps across multiple channel dimensions and suppress some unimportant channel features.

[0095] Step 804: Based on the first feature, generate features at different levels through the feature pyramid module.

[0096] In this step, the FPN module upsamples the first feature at different layers (P2 and P3 layers) to generate features at different levels. Specifically, at each layer P... n By upsampling, the feature map is enlarged, and its size is doubled.

[0097] For example, features at different levels can include high-level features and low-level features. High-level features have strong semantic information representation capabilities but low feature map resolution and weak geometric information representation capabilities. Low-level features, on the other hand, have strong geometric detail information representation capabilities and high feature map resolution but weak semantic information representation capabilities.

[0098] Step 805: Fuse features from different levels to obtain image features.

[0099] In this step, each layer C can be processed. n Feature maps after 1 After changing the number of channels through convolution 1, the resulting feature map is connected to features P at different scales via lateral connections. n The features are then added together. Thus, by upsampling and feature fusion (Feature Fusion) of feature maps from all or some layers across different levels, fused image features are obtained. The FPN module can be used to achieve feature map fusion; generally, the feature map of the last layer, P1, can be directly used as the fused feature for subsequent implicit lane representation prediction. For example, a prediction result Pred1 can be generated based on the feature map of layer P1 through the prediction module.

[0100] In some optional implementations, the AFR submodule can also collect low-level features from different layers to obtain auxiliary fusion features. For example, the AFR submodule can further collect features from layers P2 and P3 to obtain auxiliary fusion features F2 and F3. F2 and F3 are upsampled features from layers P2 and P3, respectively, and have the same size as the features from layer P1, for example, 40×100. Then, the auxiliary fusion features F2 and F3 are concatenated with features from layer P1 to obtain cross-layer fused image features for subsequent prediction. (Reference) Figure 7 As shown, based on cross-layer fusion of image features, the prediction module generates the final implicit lane prediction Pred1, which has a size of N×M×w. Here, N is the number of ELMs, M is the height of the feature map (actually the number of sampling points along the lane line longitudinally), and w is the width of the feature map. By using the AFR submodule to collect and stitch features from different layers, more contextual information can be captured, which is beneficial for the model to better understand global and local contextual information.

[0101] Step 806: Determine the horizontal set function of lane lines in the image based on image features.

[0102] In this step, lane lines in the image can be determined based on the fused image features using the following formula. level set function :

[0103] in, and Lane lines The left and right sides, d ( x, y Points with the same y-coordinate x, y ) to lane line Distance in the x-direction.

[0104] Step 807: Generate the initial implicit function based on the level set function.

[0105] refer to Figure 8b As shown, in this step, based on the level set function Generate the initial implicit function according to the following formula. : , Among them, the function This is a smooth step function (heaviside function), and the initial implicit function is... It can be viewed as the initial implicit lane representation of the prediction, which is a composite function of the level set function and the smooth step function.

[0106] In some exemplary embodiments, It can be determined using the following formula:

[0107] in, The smoothing parameter in the smoothing Heaviside function controls the width of the interval from 0 to 1.

[0108] In some exemplary embodiments, It can be approximated by the sigmoid function or the softmax function in neural networks.

[0109] Step 808: Optimize the initial implicit function to obtain the elastic lane map.

[0110] In this step, the initial implicit function (i.e., the initial implicit lane representation) can be optimized based on the prediction results of CSN and RSN to obtain the optimized target implicit function, which is then used as the final elastic lane map.

[0111] refer to Figure 7 As shown, in an exemplary embodiment, the outputs of CSN and RSN are multiplied by the initial implicit lane representation described above to obtain the final flexible lane map.

[0112] In practical applications, the initial implicit lane representation may contain an extra lane line or have lane end offsets, leading to inaccurate predictions. Based on the presence of lane lines in the image predicted by the CSN submodule, portions classified as laneless are discarded. Based on the longitudinal extension range of lane lines in the image predicted by the RSN submodule, portions exceeding the lane range are discarded. After optimizing the lane presence and lane range of the initial implicit lane representation based on the prediction results of CSN and RSN, the final elastic lane map can more accurately predict the presence and specific location of lane lines.

[0113] Combination Figure 7 As shown, when an image contains N (N is a positive integer) lane lines, N flexible lane maps can be predicted and generated. ,in, k =1, 2, ... , N.

[0114] Step 809: Determine the predicted lane lines in the image based on the zero-level profile of the elastic lane map.

[0115] In this step, the generated flexible lane map can be used as a reference. The zero-level contour line determines the predicted lane lines in the image, i.e., the elastic lane map. curves in =0 represents the first k Lane markings.

[0116] Combination Figure 8c and Figure 9 As shown, in an exemplary embodiment, based on the generated elastic lane map According to =0 Samples line by line along the y-direction to obtain the coordinates of each lane line. Specifically, this may include the following sub-steps: Sub-step 901 involves sampling the elastic lane map at equal intervals along the y-axis to obtain the coordinates of multiple target sample points located on the zero horizontal contour line of the elastic lane map.

[0117] As an exemplary implementation, the predicted lane line position can be determined by the coordinates of a set of discretized target sample points. Let M be the number of target sample points, and M be a positive integer. For the first i The coordinates of the target sample points. Here, the target sample points are the points on the y-coordinate row where the value of the elastic lane map is zero.

[0118] For example, such as Figure 8c As shown, before inputting the original image, the redundant parts of the image dataset that do not contain lane lines can be removed by cropping the image preprocessing.

[0119] Sub-step 902: Determine the predicted lane line based on the coordinates of multiple target sample points.

[0120] As an exemplary implementation, the coordinates of multiple target sample points located on the zero horizontal contour line of the elastic lane map are obtained by sampling line by line along the y-axis, and then the predicted lane line is determined based on the obtained coordinates of the multiple target sample points.

[0121] The lane detection method based on the lane detection model disclosed in the above embodiments overcomes the limitations of segmentation-based lane models (which treat lanes as long, thin objects with a finite width, typically about 30 pixels) and some line-by-line prediction methods based on coarse raster maps by combining an Elastic Lane Map (ELM) that implicitly represents lane prediction with an efficient row sampling scheme. It solves the class imbalance problem in multi-instance classification (pixel / raster) and does not require costly post-processing steps to determine lane location. Furthermore, compared to parametric and anchor-based models, ELM has stronger geometric adaptability because lane lines can have a variety of complex shapes and can bend at any angle less than 90° on the ELM.

[0122] refer to Figure 10 , Figure 10 A processing flow 1000 of a training method for a lane line detection model according to another embodiment of the present disclosure is shown. In this embodiment, using... Figure 7 Taking the lane detection network ElasticLaneNet shown as an example, process 1000 may include the following steps: Step 1001: Input the sample image into the lane line detection model.

[0123] In this step, the sample image is input into the lane detection model to be trained, for example... Figure 7 The lane detection network ElasticLaneNet is shown in the image.

[0124] Step 1002: The lane detection model outputs the initial implicit function.

[0125] In this step, the end-to-end lane detection model ElasticLaneNet is used to output the generated initial hidden function based on the input sample image.

[0126] Step 1003: Optimize the initial implicit function to obtain the optimized elastic lane map.

[0127] In this step, as an optional implementation, the initial implicit function can be optimized based on the prediction results of CSN and RSN. For example, redundant lane lines can be removed using CSN, and the portion of the initial ELM that exceeds the lane range can be reduced to zero using the RSN module, thus obtaining the optimized elastic lane map ELM.

[0128] Step 1004: Determine the EIE loss function based on the implicit function of the elastic lane map and the actual lane line labels.

[0129] As mentioned earlier, in ElasticLaneNet, predicted lane lines are represented as zero-level contours of the ELM. To learn more accurate elastic lane maps, ElasticLaneNet is trained by constructing an EIE loss function, enabling the ELM to capture the order, shape, and position of various complex geometries, thereby identifying the accurate location of lane lines.

[0130] In the lane detection problem, the predicted lane line and the true trajectory (GT) can be likened to two defect lines in a crystal dislocation. A set of curves... System elastic interaction energy It can be defined as follows:

[0131] Where, vector dl Indicates having tangential direction τ curve The line element on, that is dl = τdl , This represents a curve with another parameter. It is a curve Points on and curve Points on The Euclidean distance between them, that is, .

[0132] In ElasticLaneNet, lane lines are defined as... ,in and These are the actual labels and predicted values ​​of the lane lines, respectively. Then, the elastic interaction energy of the system described above... It can be defined by the following formula (1): (1)

[0134] In the above formula (1), energy It can be represented as Among them, self-energy That is, predicting lane lines and real labels The self-energy of each is composed of the first two terms of formula (1); while the third term of formula (1) is composed of two curves. and Interaction energy between .

[0135] Among them, the predicted lane lines and real labels Elastic interaction energy between Ei One important feature is that when predicting lane lines... and real labels When they are in opposite directions, Will tend to To minimize the total energy in formula (1), and this attractive interaction is long-range.

[0136] from Figure 11 It can be seen intuitively that when predicting lane lines... and real labels When they match but are in opposite directions, It will be completely eliminated, that is At this point, the minimum value of the elastic interaction energy in formula (1) is 0. Furthermore, the predicted lane lines... Self-energy The tendency is to smooth it out because non-smooth curves are longer and have greater self-energy.

[0137] In the embodiments of this disclosure, based on the optimized elastic lane map ELM described above, the elastic interaction energy (EIE) loss function is determined by the following formula (2). : (2) In the above formula (2), For the implicit function of the true lane line label (GT), , A horizontal set function for the actual lane line labels; For the predicted elastic lane map, 0 . 5; This is a hyperparameter.

[0138] Here, we can use formula (1) and Obtain the energy descent gradient direction of the points on the true label. and the direction of energy descent gradient at points on the predicted lane line .in Represents curve The Delta function, It is the direction of its normal. Through... and hyperparameters Control Predictive Lane Line and real labels The elastic interaction energy between them.

[0139] It should be noted that formula (2) "in The symbol ensures that they are respectively by and The indicated predicted lane lines and real labels The directions are opposite.

[0140] Step 1005: Update the parameters of the lane detection model based on the EIE loss function to obtain the updated lane detection model.

[0141] In this step, the parameters of the lane detection model can be updated based on the determined EIE loss function. For example, the EIE loss function can be used as the target loss function to train the lane detection model. When the target loss function converges, the training of the lane detection model can be stopped, and the trained lane detection model can be used as the updated lane detection model.

[0142] During training, guided by the EIE loss function, a long-range interaction will occur between the predicted lane lines and the actual trajectory, thereby guiding the curve implicitly represented on the ELM to be closer to the actual trajectory.

[0143] like Figure 11 As shown, the predicted lane lines For dynamic curves, The red arrows in the image indicate the direction of the interaction forces acting on the dynamic curve, and the lane markings are the actual labels. Indicates from dynamic curve On the real label exist Distance in direction (i.e., to) (Distance between points on the same y-coordinate). It can be seen that the elastic interaction between the two curves provides a strong attraction, and this interaction is long-range because the energy density is inversely proportional to the distance between them and decays very slowly with increasing distance. Smooth Heaviside function. It serves to smooth out dynamic curves.

[0144] In an exemplary implementation, to reduce the cost of gradient computation, FFT (Fast Fourier Transform) is applied to efficiently compute the EIE loss function. gradient value:

[0145] in, and It is the frequency in Fourier space. yes Fast Fourier Transform, This is the inverse Fourier transform.

[0146] By employing FFT, the computational cost compared to direct integration is... O ( N 2 This can reduce computing costs to O ( N log N This can significantly reduce the cost of gradient computation.

[0147] In some alternative implementations, an auxiliary loss function can also be constructed based on the auxiliary fusion features obtained from the AFR submodule. The goal of assisting in the training of the lane line detection model is to further improve the accuracy of lane line detection through cross-layer feature fusion and deep supervision during the training process.

[0148] refer to Figure 12 As shown, by constructing an auxiliary loss function The process of assisting in the training of the lane detection model may include the following sub-steps: Sub-step 1201: Feature collection is performed through the AFR sub-module to obtain auxiliary fusion features.

[0149] The AFR submodule can further collect partial hierarchical features from different layers to obtain auxiliary fusion features. For example, by upsampling layers P2 and P3, auxiliary fusion features F2 and F3 are obtained.

[0150] Sub-step 1202: Based on the auxiliary fusion features, generate auxiliary prediction results through the prediction module.

[0151] Please see Figure 7 Based on the auxiliary fusion features F2 and F3 obtained in sub-step 1201, auxiliary prediction results Pred2 and Pred3 are generated by the prediction module respectively.

[0152] Sub-step 1203: Determine the auxiliary loss function based on the auxiliary prediction results and the actual lane line labels.

[0153] As an exemplary implementation, an auxiliary loss function is composed of EIE loss. The definition is as follows:

[0154] in, For parameters, ; and Pred2 and Pred3 are the prediction results from layers P2 and P3 of the FPN module, respectively.

[0155] Sub-step 1204: Update the parameters of the lane detection model based on the auxiliary loss function to obtain the updated lane detection model.

[0156] As an exemplary implementation, an auxiliary loss function composed of EIE loss is used. Deep supervised learning is performed on Pred1, Pred2, and Pred3. The Pred1 obtained after deep supervised learning is then multiplied by the prediction results of CSN and RSN to obtain the final Elastic Lane Map (ELM).

[0157] Furthermore, in an exemplary implementation, the existence and extent (length) of each lane line in the ELM can be jointly learned by constructing a corresponding loss function based on the prediction results of the CSN and RSN submodules. Specifically, the CSN submodule is used to predict whether lane lines exist in the image, and the RSN submodule is used to predict the longitudinal extension extent (along the y-axis) of lane lines in the image.

[0158] As an exemplary implementation, based on the predicted probability of the existence of each lane line predicted by the CSN submodule, a lane existence loss function is defined. Using lane existence loss function Update the parameters of the CSN submodule. Lane existence loss function. The focal loss function can be used, which is defined as follows:

[0159] in, Y i In the one-hot representation of whether or not actual lane line labels exist, Y i It can be 0 or 1; P i It is the first i The predicted probability of the existence of lane markings. For example, the parameter... and γIt can be set to 0.25 and 2 respectively.

[0160] As an exemplary implementation, the RSN submodule's parameters are updated by defining a lane range loss function based on the predicted probability of lane lines existing in each y-coordinate row of the image predicted by the RSN submodule. For example, the lane range loss function can be determined based on the predicted probability of lane lines existing in each y-coordinate row of the sample image and the binary cross-entropy of the one-hot representation of the actual lane line label range. By applying this lane range loss function Learn every line y i The existence of a specific lane is determined to obtain the lane range. Lane range loss function. It can be defined as follows:

[0161] in, y i For the first i The unique and real label of the lane markings. p ( y i ) for the first i The predicted positive class probability on the row (i.e., the first row) i The probability that a row contains lane lines), M is the number of rows going up along the y-axis.

[0162] In the above exemplary embodiments, the CSN submodule can utilize the lane existence loss function for cases without lane markings. Training is performed; occasionally, lane endpoints on the predicted ELM will deviate. The RSN submodule can replace the post-processing step to eliminate outliers, discarding parts that are outside the lane range or classified as laneless, thus obtaining an optimized ELM.

[0163] Furthermore, as an exemplary implementation, the auxiliary loss function described above can also be used. Lane existence loss function Lane range loss function and EIE loss function Determine the total loss function, and jointly update the parameters of the AFR submodule, CSN submodule, RSN submodule, and prediction module based on the total loss function. Total Loss Function It can be defined as follows:

[0164] in, , , and These are the weighting coefficients for each loss term.

[0165] As described above, the training method for the lane detection model provided in this disclosure uses an energy functional (EIE) loss specifically designed for sparse lane points to achieve global context integration and ELM learning. The ELM evolves gradually based on the EIE loss during training, generating a long-range interaction between the lane prediction result and the actual lane trajectory, thereby guiding the curve implicitly represented on the ELM to approach the actual trajectory.

[0166] In summary, the embodiments of this disclosure propose a lane detection model, ElasticLaneNet, with flexible geometric adaptability, which excels at detecting lanes with complex geometries while maintaining high efficiency. Within the ElasticLaneNet framework, a novel implicit lane representation method, namely the zero-level contour line of ELM, is designed, enabling the lane detection method based on the model to perform excellently in lane recognition for complex geometries such as intersections, various curves, Y-shaped forks, and lanes merging at large angles. By applying EIE loss to guide ELM learning, long-range lane information is integrated across the entire image space while maintaining attention to local features. Furthermore, various auxiliary branch structures are designed, such as bottleneck layers and AFR submodules, to enhance feature fusion capabilities; and ELM is optimized through joint training of CRN and RSN submodules with the prediction module. The lane detection model / method of this disclosure can accurately identify lanes with complex geometries and shapes, and provides stable lane detection results even under conditions of weak features (such as nighttime, shadows, and occlusion).

[0167] The experimental evaluation results of the lane detection method based on the lane detection model of this disclosure on different datasets will be described in detail below.

[0168] This disclosure proposes a novel lane line dataset, SDLane, and applies the lane line detection method based on the lane line detection model of this disclosure to the following three datasets for experiments: SDLane, CULane, and TuSimple. Experimental results show that the method of this disclosure achieves state-of-the-art performance on the SDLane dataset, and also performs well on the other two mainstream lane line datasets, CULane and TuSimple, with fast inference speed and high FPS (Frames Per Second).

[0169] SDLane is a newly proposed dataset with up to 7 lanes and various highly complex lane structures, including various intersections, Y-shaped lanes or merging lanes with different angles, and dense lanes. CULane categorizes driving scenarios into 9 classes with up to 4 lanes, including some complex scenarios such as dense lanes and nighttime scenarios. Notably, in SDLane, any drivable lane, such as the intersecting lanes at an intersection, is considered a lane, while in CULane, intersections are considered laneless. TuSimple is a highway dataset for good weather conditions with up to 5 lanes. The curve ratio in the SDLane dataset is greater than 90%, the curve ratio in the CULane dataset is approximately 2%, and the curve ratio in the TuSimple dataset is approximately 30%.

[0170] Evaluations on the CULane and SDLane datasets can use CULane's official evaluation metrics: F1 score based on IoU (Intersection over Union), precision, and recall, where: F1=2 (Precision Recall) / (Precision+Recall); Precision = TP / (TP + FP); Recall = TP / (TP + FN).

[0171] Wherein, TP represents the number of accurately predicted lane points, that is, the number of samples where the model predicts positive and the actual value is positive; FP represents the number of samples where the model predicts positive but the actual value is negative; and FN represents the number of samples where the model predicts negative but the actual value is positive.

[0172] The official evaluation metrics for the TuSimple dataset are accuracy (Acc), first precision (FP), and second nearest non-precision (FN). Additionally, the F1 score can also be used as an evaluation metric. Here, accuracy (Acc) = N. pred / N gt N pred To accurately predict the number of lane points, N gt The number of lane points labeled with actual lane lines.

[0173] Furthermore, during the experiment, the total loss function Parameters in , , and The values ​​are set to 1.0, 1.0, 0.1, and 0.2, respectively. For the case where feature fusion does not have an AFR submodule, the auxiliary loss function... Parameters in and All values ​​are set to 0.3, and all other values ​​are set to 0. This means that the multi-scale splicing feature fusion method and the auxiliary loss function are two further feature fusion methods, and they are not used together.

[0174] refer to Figure 13 , Figure 13 The rows in the figure show a variety of challenging and complex scenarios from the SDLane dataset. The top row 1 shows a winding road scenario, the second row shows a right-turn branch obscured by vehicles, the third row shows a left-turn branch obscured by vehicles, the fourth row shows a three-way intersection, the fifth row shows a newly added left-hand branch lane, the sixth row shows a Y-shaped intersection lane with a large curvature left turn and a large angle, and the bottom row 7 shows an intersection lane in a wireless scenario.

[0175] Figure 13 The columns in this document show a comparison of the prediction results of the ElasticLaneNet lane detection network of this disclosure with other lane detection models on the structurally diverse SDLane dataset. Column 1 represents the input sample image, column 2 represents the ground truth lane labels (GT), columns 3 through 6 represent the prediction results of different types of state-of-the-art (SOTA) lane detection models LaneAF, BezierLaneNet, CLRNet, and CondLaneNet, respectively, and column 7 represents the prediction results based on ElasticLaneNet of this disclosure. Different colors (e.g., red, green, blue, yellow) represent different instances.

[0176] Based on the state-of-the-art (SOTA) lane detection model for each type, the models were trained to convergence. The prediction results obtained through experiments are recorded in Table 1 below. In Table 1 and below, ElasticLaneNet represents data without the feature enhancement bottleneck layer (TB). T This represents data with a feature-enhanced bottleneck layer.

[0177] Table 1. Comparison of evaluation metrics of prediction results of ElasticLaneNet and other state-of-the-art lane detection models in different scenarios on the SDLane dataset.

[0178] Combination Figure 13As shown in Table 1, segmentation-based lane detection models such as LaneAF may produce incomplete predictions or missing lane lines when lane features are weak or overlapping lanes (such as Y-shaped lanes). This reflects their heavy reliance on low-level features and complex post-processing. While the parameter-based lane detection model BezierLaneNet can handle many challenging instances, its predictions deviate significantly from the ground truth (GT) when lane shapes are diverse. CondLaneNet is an effective line-by-line processing method and incorporates an auxiliary module called RIM (Recursive Instance Module) to handle bifurcated lanes. However, when faced with large-angle bifurcations or lanes with weak features, CondLaneNet often fails to provide the precise shape and location of the branch lanes. Notably, although the anchor-based lane detection model CLRNet achieves the highest accuracy (least redundant predictions) in Table 1, it often misses dense lanes, Y-shaped lanes, and turns, which are crucial for autonomous vehicles. For these scenarios, the anchor points are not flexible enough to fully represent diverse lane structures.

[0179] Table 2 below summarizes and compares the FPS inference speed of ElasticLaneNet of this disclosure and other lane detection models applied to the SDLane dataset on the same server.

[0180] Table 2. FPS comparison of ElasticLaneNet and other state-of-the-art lane detection models on the SDLane dataset

[0181] According to the summary comparison in Table 2, besides the parameter-based BezierLaneNet achieving the fastest speed (109.37 FPS), the ElasticLaneNet based on this disclosure ranked second, reaching 75.42 FPS and 66.62 FPS respectively (without / with feature enhancement bottleneck layers TB). However, combining Table 1 and... Figure 13 However, the predictions of parameter-based BezierLaneNet are not very accurate. The segmentation-based lane detection model LaneAF requires a time-consuming post-processing step, while the anchor-based lane detection model CLRNet requires NMS to select the best prediction. Furthermore, CondLaneNet, based on row prediction, requires time to integrate multiple outputs, such as confidence maps, offset maps, and starting point maps, and needs to use an RNN (Recurrent Neural Network) module in the post-processing step to handle Y-shaped lane structures. Therefore, ElasticLaneNet, based on this disclosure, achieves a better balance between accuracy and real-time efficiency.

[0182] Table 3 below shows a comparison of the prediction results of ElasticLaneNet and other mainstream state-of-the-art lane detection models based on the ResNet34 architecture on the TuSimple dataset.

[0183] Table 3. Evaluation metrics of ElasticLaneNet and other state-of-the-art lane detection models on the TuSimple dataset.

[0184] TuSimple is an early lane detection dataset that primarily includes highway driving scenarios under good lighting conditions. Comparing ElasticLaneNet with most state-of-the-art lane detection models (most of which use a ResNet34 backbone) reveals that ElasticLaneNet outperforms most existing models. Specifically, its F1 score of 97.05 and accuracy (Acc) of 96.48% are close to the performance of current state-of-the-art (SOTA) lane detection models (F1 score of 97.82 and accuracy of 96.9%).

[0185] Table 4 below shows a comparison of the prediction results of ElasticLaneNet and other state-of-the-art lane detection models on the CULane dataset. Among them, ELM- MSE indicates that the Mean Squared Error (MSE) loss function was used instead of the Elastic LaneNet's EIE loss function. The MSE loss function evaluates the model's performance by calculating the average of the squared differences between the predicted and actual values.

[0186] Table 4. Evaluation metrics of ElasticLaneNet and other state-of-the-art lane detection models on the CULane dataset.

[0187] In Table 4, the total F1 score is the F1 score based on the threshold IoU=0.5. Scenes 1 to 9 in the table correspond to the following scenes in the CULane dataset: Scene 1, Normal; Scene 2, Crowded; Scene 3, Dazzle; Scene 4, Shadow; Scene 5, Noline (i.e., no lane markings); Scene 6, Arrow; Scene 7, Curve; Scene 8, Cross; Scene 9, Night.

[0188] As shown in Table 4, based on the comparison of prediction results of ElasticLaneNet and other state-of-the-art lane detection models on the CULane dataset, ElasticLaneNet based on this disclosure also performs well on the commonly used CULane dataset, which covers a variety of scenarios, and surpasses many state-of-the-art lane detection models.

[0189] Figure 14 The results show the predictions of ElasticLaneNet in some scenarios on the CULane dataset and some challenging scenarios on the TuSimple dataset.

[0190] in, Figure 14 The first row shows the curves, dense lanes on both sides, and weak lane features under occlusion / shading conditions in the TuSimple dataset. Figure 14 The other rows show crowded scenes, nighttime scenes, wireless scenes, shadow scenes, glare scenes, and curve scenes from the CULane dataset, respectively.

[0191] Figure 15 The results show a comparison between ElasticLaneNet's predictions and the true lane label in some scenarios on the CULane dataset and some challenging scenarios on the TuSimple dataset.

[0192] Combination Figure 14 and Figure 15 As shown, ElasticLaneNet performs well in some challenging scenarios on the TuSimple dataset (such as...). Figure 14 , Figure 15 It demonstrated superior performance in the first row and achieved accurate lane line prediction results in many complex scenarios of the CULane dataset.

[0193] Since most lane lines in the CULane dataset are straight, ElasticLaneNet achieves good prediction results with only 18 sampling points while maintaining efficient inference speed. The tested inference FPS is 105 / 90 (without / with feature enhancement bottleneck layer), which is much faster than the 36 sampling points shown in Table 2. Figure 14 This demonstrates ElasticLaneNet's excellent prediction capabilities for challenging CULane scenarios. Due to the long-range characteristics guided by the EIE loss function, ElasticLaneNet excels at handling weak lane features, such as completely obscured lanes by cars or glare, completely dark nights without any reference lines, and congested traffic scenes with a large number of pedestrians and vehicles.

[0194] On the other hand, the current state-of-the-art (SOTA) methods are anchor-based methods and line-by-line prediction methods based on CGM. ElasticLaneNet's advantage on CULane is not significant, possibly because the lane structures in CULane are mostly parallel and straight, and "intersection scenarios" are treated as "laneless." However, ElasticLaneNet excels in handling the diverse lane shapes and intersections in the SDLane dataset. Furthermore, geometrically diverse models generally have greater degrees of freedom (DOF), providing richer geometric representations, but may sacrifice some accuracy; this phenomenon also exists in some works focusing on lane shape modeling. Therefore, adjusting and optimizing the CSN submodule for the characteristics of the CULane dataset is a possible direction for future improvement.

[0195] In addition, to further verify the effectiveness of the lane detection model ElasticLaneNet disclosed herein, ablation experiments were conducted on the model's network architecture and loss function.

[0196] Table 5 below shows the overall ablation experiments of the ElasticLaneNet network architecture on the SDLane dataset.

[0197] Table 5. Ablation experiments of ElasticLaneNet on the SDLane dataset

[0198] In Table 5, the evaluation indicators for the ablation experiment in the first row are: ELM- MSE refers to data where the MSE loss function replaces the EIE loss function of ElasticLaneNet. In other words, it involves simple post-processing to remove outliers that exceed a predetermined threshold. Compared to the second line, the EIE loss function significantly improves various evaluation metrics.

[0199] As shown in Table 5, refining the ELM process by replacing the post-processing flow with CSN and RSN submodules can improve the F1 score and accuracy, but at the cost of some recall. That is, using CSN / RSN submodules makes the prediction results more conservative. If the prediction results of the CSN / RSN submodules are incorrect, especially in challenging scenarios (such as those with dense lanes on both sides), the model will lose some detection capability. In contrast, the AFR submodule, with its cross-layer feature collection and feature fusion scheme, improves overall performance.

[0200] As shown in Tables 1, 3 and 5, the design of the bottleneck layer brings about a significant performance improvement, presumably because the self-attention mechanism improves the prediction accuracy of lane presence and lane range, both of which are very beneficial to the refinement of ELM.

[0201] In addition, to investigate a better lane characterization method, a variant of the prediction module (i.e., ELMM) as the detection head was also examined in the ablation experiments. This variant is an explicit point-by-point prediction detection head that does not include the FPN module and instead uses upsampling.

[0202] like Figure 16 As shown, embodiments of this disclosure provide a lane detection model, referred to as the explicit model ElasticLaneNet. pw The lane detection model can include a feature extraction network and a prediction module.

[0203] Corresponding to the explicit model ElasticLaneNet mentioned above pw The embodiments of this disclosure also provide a lane line detection method based on a lane line detection model that is similar to the lane line detection method in the foregoing embodiments. The lane line detection method may include: extracting image features of the image to be processed through a feature extraction network; and determining the predicted lane lines in the image to be processed through a prediction module based on the image features, wherein the predicted lane lines are represented as the derivative of the implicit function of the lane lines in the image to be processed.

[0204] Furthermore, embodiments of this disclosure also provide the above-described lane detection model, ElasticLaneNet. pw The training method may include: extracting image features of sample images through a feature extraction network; determining predicted lane lines in the image to be processed through a prediction module based on the image features, wherein the predicted lane lines are represented as the derivative of the implicit function of lane lines in the image to be processed; determining the elastic interaction energy loss function based on the predicted lane lines and the real lane line labels of the sample images; and updating the parameters of the lane line detection model based on the elastic interaction energy loss function to obtain the updated lane line detection model.

[0205] refer to Figure 16 As shown, for a given image I∈R H×W×3 ElasticLaneNet pw The prediction module can directly output predicted lane lines, which are represented by a set of x-coordinates. For example, to output N predicted lane lines, ElasticLaneNet... pw The network output is O ' ∈R N×M Among them, network output O'Can be represented as follows:

[0206] in, k =1, 2, ..., N, where N is the number of lane lines predicted.

[0207] Compared to ElasticLaneNet's ELM-based implicit representation of zero-level contour lines for lane line prediction, ElasticLaneNet... pw It can directly output the x-coordinates of N predicted lane lines, i.e. Lane line real labels The coordinates of the point on are .

[0208] As mentioned above, the elastic interaction energy of a pair of open curves E It can be defined as:

[0209] In embodiments of this disclosure, the explicit model ElasticLaneNet pw The elastic interaction energy between the predicted lane line and the actual lane line E It can be represented as follows:

[0210] In the above formula, The Delta function is the same as the aforementioned Heaviside function. The derivative of .

[0211] To simplify calculations, the following regularization Delta function is used:

[0212] This is An approximation of the derivative of; where, To predict the x-coordinate of a point on the lane line, ; For smoothing parameters.

[0213] From the above, it can be seen that the following conditions are met. Therefore, the explicit model ElasticLaneNet pw The EIE loss function between the predicted lane line and the actual lane line The same formula (2) above can be used to determine it. Furthermore, it can be based on this EIE loss function. For the explicit model ElasticLaneNet pw The parameters are updated to obtain the updated lane detection model.

[0214] Among them, the explicit model ElasticLaneNet pw The difference between ElasticLaneNet and the aforementioned ElasticLaneNet is that ElasticLaneNet outputs an elastic lane map and determines the EIE loss function based on the elastic lane map; while the explicit model ElasticLaneNet... pw The predicted lane line is directly output, which is represented as the derivative of the implicit function, i.e., the Delta function. When determining the EIE loss function, the implicit function is first determined based on the Delta function, and then the EIE loss function is determined based on the implicit function.

[0215] refer to Figure 16 As shown, in an exemplary implementation, the explicit model ElasticLaneNet pw This can include a CSN submodule and an RSN submodule. When determining the EIE loss function, the predicted lane line output by the prediction module can be multiplied with the outputs of CSN and RSN to optimize the predicted lane line. Based on the optimized predicted lane line, an initial implicit function is determined, and the EIE loss function is then determined based on this implicit function. The roles and effects of the CSN and RSN submodules are described in the preceding embodiments and will not be repeated here.

[0216] Explicit network mapping is trained by minimizing the EIE loss. ElasticLaneNet pw The evaluation metrics for the prediction results in different datasets are shown in Table 6 below.

[0217] Table 6. Explicit Model ElasticLaneNet pw Evaluation indicators

[0218] By using the explicit model ElasticLaneNet with point-by-point predictions in Table 6 pw Compared with the evaluation metrics of ElasticLaneNet with ELMM detection heads in Table 1, ElasticLaneNet pw Its advantages lie in its smaller model parameters and faster inference speed. Among them, ElasticLaneNet... pw It performs significantly worse than ElasticLaneNet on structurally complex datasets (such as SDLane), but is not much different on CULane and TuSimple, indicating that the explicit model ElasticLaneNet... pw Predictive potential on lanes with simple structures.

[0219] Furthermore, to demonstrate the effectiveness of the EIE loss, another classic loss function, the MSE loss function, was applied to supervised learning of ELM in the ablation experiments. The performance of ELM trained using the MSE loss function and the EIE loss function on different datasets is shown in Table 7 below.

[0220] Table 7. Comparison of ELM trained based on MSE loss function and EIE loss function

[0221] According to Table 7, compared to the MSE loss function, the EIE loss function is more effective on the scene-diverse dataset CULane and the structurally complex dataset SDLane. Furthermore, in experiments on the TuSimple dataset, the predictions trained using the EIE loss function have smaller FP and FN values, resulting in higher accuracy. Acc higher.

[0222] In addition, the impact of the MSE loss function on the explicit model ElasticLaneNet was also examined. pw Training of ElasticLaneNet. pw The performance is acceptable; however, without prior anchors or CGM, using only MSE loss in ElasticLaneNet is problematic. pw The experiment may fail due to training failure.

[0223] By applying the lane detection method based on the lane detection model disclosed herein to three datasets: SDLane, TuSimple, and CULane, experimental results show that the method outperforms existing models on the structurally diverse SDLane dataset, achieving state-of-the-art (SOTA) performance with an F1 score of 89.51, a recall of 87.50, and a precision of 91.61, while maintaining fast inference speed (this result is achieved without using a feature enhancement bottleneck layer). Ablation experiments demonstrate the functionality of different modules in the ElasticLaneNet network and the EIE Loss-guided training strategy, making the ELM used for implicit lane representation more effective and accurate than explicit lane representation methods when facing challenging cases.

[0224] The following is for reference. Figure 17 It illustrates a device suitable for implementing embodiments of the present disclosure (e.g., Figure 3The diagram shows the structure of a computer system 700 (including servers or terminal devices). Terminal devices in the embodiments of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0225] like Figure 17 As shown, the computer system 700 may include a processor (e.g., CPU, Central Processing Unit) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0226] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0227] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined above in the methods of embodiments of this disclosure.

[0228] It should be noted that the computer-readable medium described in the embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the embodiments of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0229] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, can perform the various methods and processes described above, such as lane line detection methods based on lane line detection models and training methods for lane line detection models.

[0230] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0231] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0232] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The described modules can also be located in a processor, and the names of these modules do not necessarily limit the module itself.

[0233] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A lane line detection method based on a lane line detection model, wherein the lane line detection model includes a feature extraction network and a prediction module, and the method includes: The feature extraction network extracts image features from the image to be processed. Based on the image features, the prediction module determines the initial latent function of the lane lines in the image to be processed, and generates an elastic lane map based on the initial latent function; The predicted lane lines in the image to be processed are determined based on the zero-level contour line of the elastic lane map.

2. The lane line detection method according to claim 1, wherein the initial implicit function of the lane line is a composite function of a horizontal set function and a smooth step function.

3. The method according to claim 2, wherein, Determining the predicted lane lines in the image to be processed based on the zero-level contour line of the elastic lane map includes: On the elastic lane map, samples are taken at equal intervals along the y-axis to obtain the coordinates of multiple target sample points located on the zero horizontal contour line of the elastic lane map, wherein the target sample points are points on the y-coordinate row of the elastic lane map with a value of zero. The predicted lane line is determined based on the coordinates of the multiple target sample points.

4. The lane line detection method according to claim 1, wherein, The feature extraction network includes a feature extraction subnetwork and a multi-scale feature fusion subnetwork. The feature extraction subnetwork includes a residual module. The step of extracting image features from the image to be processed through the feature extraction network includes: The residual module extracts the first feature from the image to be processed. Based on the first feature, the image features are obtained by feature fusion through the multi-scale feature fusion sub-network.

5. The lane line detection method according to claim 1, wherein, The feature extraction network includes a feature extraction subnetwork and a multi-scale feature fusion subnetwork. The feature extraction subnetwork includes a residual module and a feature enhancement bottleneck layer. The step of extracting image features from the image to be processed through the feature extraction network includes: The initial features are extracted from the image to be processed by the residual module; the initial features are then globally enhanced by the feature enhancement bottleneck layer to obtain the first feature; based on the first feature, the image features are obtained by feature fusion through the multi-scale feature fusion sub-network.

6. The lane line detection method according to claim 4 or 5, wherein, The multi-scale feature fusion subnetwork includes a feature pyramid module and an auxiliary feature optimization submodule. The step of fusing features based on the first feature through the multi-scale feature fusion sub-network to obtain the image features includes: Based on the first feature, multiple features at different levels are generated through the feature pyramid module and fused to obtain the fused feature. For features at at least one of the multiple different levels, auxiliary fusion features are obtained by feature collection through the auxiliary feature optimization submodule, and the auxiliary fusion features are concatenated with the fusion features to obtain the image features.

7. The lane line detection method according to claim 4 or 5, wherein, The lane line detection model also includes a lane presence classification subnetwork and a lane range subnetwork. The generation of the elastic lane map based on the initial implicit function includes: Based on the first feature, the presence of lane lines in the image to be processed is predicted by the lane existence classification subnetwork, and the longitudinal extension range of lane lines in the image to be processed is predicted by the lane range subnetwork. Based on the lane existence classification subnetwork and the prediction results of the lane existence classification subnetwork, the lane existence and lane range of the initial implicit function determined by the prediction module are optimized, and the optimized target implicit function is determined as the generated elastic lane map.

8. A training method for a lane line detection model, wherein, The lane detection model includes a feature extraction network and a prediction module, and the method includes: Image features of the sample images are extracted using the feature extraction network; Based on the image features, the prediction module determines the initial latent function of the lane lines in the sample image, and generates an elastic lane map based on the initial latent function; The elastic interaction energy loss function is determined based on the implicit function of the real lane line labels in the elastic lane map and the sample image; The lane detection model is updated by updating its parameters based on the elastic interaction energy loss function, resulting in an updated lane detection model.

9. The training method according to claim 8, wherein updating the parameters of the prediction module based on the elastic interaction energy loss function comprises: The gradient value of the elastic interaction energy loss function is determined in Fourier space by using Fast Fourier Transform; The lane detection model is updated with parameters based on the determined gradient value.

10. The training method according to claim 8, wherein, The feature extraction network includes a feature extraction subnetwork and a multi-scale feature fusion subnetwork. The feature extraction subnetwork includes a residual module and a feature enhancement bottleneck layer. The multi-scale feature fusion subnetwork includes a feature pyramid module and an auxiliary feature optimization submodule. The step of extracting image features from the sample image through the feature extraction network includes: The residual module extracts the initial features from the sample image; The initial feature is enhanced globally through the feature enhancement bottleneck layer to obtain the first feature; Based on the first feature, multiple features at different levels are generated through the feature pyramid module and fused to obtain the fused feature. For features at at least one of the multiple different levels, auxiliary fusion features are obtained by feature collection through the auxiliary feature optimization submodule, and the auxiliary fusion features are concatenated with the fusion features to obtain the image features.

11. The training method according to claim 10, wherein, The method further includes: Based on the aforementioned auxiliary fusion features, the prediction module generates auxiliary prediction results. An auxiliary loss function is determined based on the auxiliary prediction results and the real lane line labels, and the parameters of the lane line detection model are updated based on the auxiliary loss function to obtain the updated lane line detection model. The auxiliary loss function is determined based on the energy loss from elastic interactions.

12. The training method according to claim 11, wherein the lane line detection model further includes a lane presence classification subnetwork, and the method further includes: Based on the first feature, the lane presence classification subnetwork is used to predict the probability of each lane line in the sample image. The lane presence loss function is determined based on the predicted probability of each lane line in the sample image and the focal loss function of the one-hot representation of the presence or absence of the actual lane line label. The parameters of the lane presence classification subnetwork are updated based on the lane presence loss function.

13. The training method according to claim 12, wherein, The lane line detection model further includes a lane range sub-network, and the method further includes: Based on the first feature, the lane range sub-network is used to predict the probability of lane lines existing in each y-coordinate row of the sample image. The lane range loss function is determined based on the predicted probability of lane lines existing in each y-coordinate row of the sample image and the cross-entropy of the one-hot representation of the actual lane line label range. The parameters of the lane range subnetwork are updated based on the lane range loss function.

14. The training method according to claim 13, wherein, The parameter update of the lane detection model based on the elastic interaction energy loss function includes: The total loss function is determined based on the auxiliary loss function, the lane existence loss function, the lane range loss function, and the elastic interaction energy loss function. Based on the total loss function, the parameters of the auxiliary feature optimization submodule, the lane existence classification subnetwork, the lane range subnetwork, and the prediction module are jointly updated.

15. A lane line detection method based on a lane line detection model, wherein the lane line detection model includes a feature extraction network and a prediction module, and the method includes: The feature extraction network extracts image features from the image to be processed. Based on the image features, the prediction module determines the predicted lane lines in the image to be processed, wherein the predicted lane lines are represented as the derivative of the implicit function of the lane lines in the image to be processed.

16. A method for training a lane detection model, the lane detection model comprising: The method includes a feature extraction network and a prediction module, comprising: Image features of the sample images are extracted using the feature extraction network; Based on the image features, the prediction module determines the predicted lane lines in the image to be processed, wherein the predicted lane lines are represented as the derivative of the implicit function of the lane lines in the image to be processed. The elastic interaction energy loss function is determined based on the predicted lane lines and the real lane line labels of the sample images; The lane detection model is updated by updating its parameters based on the elastic interaction energy loss function, resulting in an updated lane detection model.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the lane detection method based on the lane detection model according to any one of claims 1-7 and 15, or the training method of the lane detection model according to any one of claims 8-14 and 16.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the lane detection method based on the lane detection model according to any one of claims 1-7 and 15, or the training method of the lane detection model according to any one of claims 8-14 and 16.

Images