An end-to-end lane line detection method, system, device, and medium

Abstract

The application provides an end-to-end lane line detection method, system, device and medium, comprising: acquiring a front road surface image of a vehicle; inputting the front road surface image into a feature extraction network to obtain a feature map; inputting the feature map into a pre-trained parameter prediction network to obtain lane line parameters of each lane line in the road surface image, the lane line parameters being used to represent the positional relationship of each point of the same lane line; and determining the geometric representation of each lane line in the front road surface image according to the lane line parameters. The application can realize complete end-to-end lane line detection, is convenient for being embedded into a vehicle end application to assist vehicle driving, and ensures driving safety.

Description

Technical Field

[0001] The present application relates to the field of vehicle-mounted recognition applications, and in particular to an end-to-end lane line detection method, system, device, and medium. Background Art

[0002] Computer vision methods have been widely used in autonomous and assisted driving systems, with lane detection technology being a fundamental and crucial component. Many traffic accidents are caused by unclear lane markings. Improving lane detection capabilities in assisted driving technology can significantly reduce the loss of life and property in these situations. Lane detection involves using various computer-assisted methods to automatically and accurately detect yellow and white lines on the road, providing complete lane information to the driver and providing input for features such as lane keeping and adaptive cruise control.

[0003] Existing lane line detection methods rely on multiple neural networks connected in series. Network training is difficult and has high requirements for the construction of training samples. It requires a large amount of computing resources and is difficult to meet the application scenario requirements of vehicle-side embedding. Summary of the Invention

[0004] In view of the above problems existing in the existing technology, this application proposes an end-to-end lane line detection method, system, equipment and medium, which mainly solves the problems of complex lane line detection network structure, high resource consumption, and difficulty in achieving end-to-end vehicle-side data recognition.

[0005] In order to achieve the above-mentioned objectives and other objectives, the technical solutions adopted in this application are as follows.

[0006] This application provides an end-to-end lane detection method, including:

[0007] Acquire a forward road image of the vehicle;

[0008] Inputting the forward road surface image into a feature extraction network to obtain a feature map;

[0009] Inputting the feature map into a pre-trained parameter prediction network to obtain lane line parameters of each lane line in the road surface image, wherein the lane line parameters are used to represent the positional relationship between each point on the same lane line;

[0010] A geometric representation of each lane line in the front-view road surface image is determined according to the lane line parameters.

[0011] In one embodiment of the present application, the pre-training step of the parameter prediction network includes:

[0012] Label the lane lines in the preset sample image to obtain training samples;

[0013] Inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain parameter prediction results and lane line prediction results;

[0014] Determining a total network loss value based on the parameter prediction results and the lane line prediction results, and updating network parameters based on the total network loss value until the total loss value reaches a preset threshold, thereby completing pre-training of the parameter prediction network;

[0015] After completing the pre-training of the parameter prediction network, the lane line prediction network branch is discarded.

[0016] In one embodiment of the present application, determining the total network loss value according to the parameter prediction result and the lane line prediction result includes:

[0017] Performing lane line fitting according to the parameter prediction result to obtain a fitted lane line, wherein the parameter prediction result includes a first loss value of the parameter prediction network;

[0018] Using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch based on a similarity between the lane line prediction result and the label;

[0019] The first loss value and the second loss value are added together to obtain the total network loss value.

[0020] In one embodiment of the present application, the network structure of the feature extraction network is, from left to right, a slicing module, a first convolution module, a first residual module, a second convolution module, and a second residual module;

[0021] The forward road image sequentially passes through the segmentation module, the first convolution module, the first residual module, the second convolution module, and the second residual module to obtain a corresponding feature map; wherein the first convolution module and the second convolution module have the same structure.

[0022] In one embodiment of the present application, the slicing module includes a slicing layer, a first feature splicing layer, and a first convolution layer, and the front view road image passes through the slicing layer, the first feature splicing layer, and the first convolution layer in sequence to obtain a compressed feature map;

[0023] The first convolution module includes a second convolution layer, a first batch normalization layer, and a first activation function layer, and the compressed feature map passes through the second convolution layer, the first batch normalization layer, and the first activation function layer in sequence to obtain the output of the first convolution module;

[0024] The first residual module includes a first branch, a second branch, a second feature splicing layer, a second batch normalization layer, a second activation function layer, a third convolution layer, a third batch normalization layer and a third activation function layer, the first branch includes a convolution layer, and the second branch is composed of a fourth convolution layer, a fourth batch normalization layer, a fourth activation function layer, a residual network unit and a fifth convolution layer connected in series; the output of the first convolution module is processed by the first branch and the second branch respectively, and the feature maps output by the first branch and the second branch are spliced ​​through the second feature splicing layer, and then processed in sequence by the second batch normalization layer, the second activation function layer, the third convolution layer, the third batch normalization layer and the third activation function layer to obtain the output of the first residual module;

[0025] The second residual module has a different number of residual network units from the first residual module, but the rest of the structure is the same. When there are multiple residual network units, each residual network unit is connected in series in sequence.

[0026] In one embodiment of the present application, the lane line prediction network branch includes at least: three downsampling layers, a first fully connected layer and a loss function layer; the three downsampling layers each obtain a feature map of a different dimension, the output of each downsampling layer is connected to the input of the first fully connected layer, the feature maps of different dimensions are fused through the first fully connected layer, and then the loss value after feature fusion is calculated through the loss function layer.

[0027] In one embodiment of the present application, the network structure of the parameter prediction network includes, from left to right: a sixth convolutional layer, a seventh convolutional layer, a feature compression layer, a second fully connected layer, and a third fully connected layer;

[0028] The number of neurons in the third fully connected layer is equal to the number of lane line parameters to be predicted.

[0029] In one embodiment of the present application, the convolution kernels of the sixth convolution layer and the seventh convolution layer adopt 3*3 convolution kernels, and the moving step size of the convolution kernel is 2.

[0030] In one embodiment of the present application, the geometric representation of the lane line is represented by a cubic curve.

[0031] This application also provides an end-to-end lane detection system, including:

[0032] An image acquisition module, used to acquire a front view road image of the vehicle;

[0033] A feature extraction module, configured to input the forward road surface image into a feature extraction network to obtain a feature map;

[0034] a lane line parameter prediction module, configured to input the feature map into a pre-trained parameter prediction network to obtain lane line parameters for each lane line in the road surface image, wherein the lane line parameters are used to represent the positional relationship between points on the same lane line;

[0035] A lane line representation module is used to determine the geometric representation of each lane line in the front view road surface image according to the lane line parameters.

[0036] The present application also provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the end-to-end lane line detection method when executing the computer program.

[0037] The present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the end-to-end lane line detection method when executed by a processor.

[0038] As described above, the present application provides an end-to-end lane detection method, system equipment, and medium, which have the following beneficial effects.

[0039] This application obtains a forward-view road image of a vehicle; inputs the forward-view road image into a feature extraction network to obtain a feature map; inputs the feature map into a pre-trained parameter prediction network to obtain lane line parameters for each lane line in the road image; and determines the geometric representation of each lane line in the forward-view road image based on the lane line parameters. This application only requires inputting the features of the forward-view road image collected by the vehicle into a pre-trained parameter prediction network to obtain the prediction results of the lane line parameters, and obtains a collective representation of the lane lines based on the lane line parameter fitting. It does not require multiple deep neural networks to execute the prediction process in series, which greatly simplifies the model structure. The lane line prediction results are obtained by inputting the road image, realizing a complete end-to-end detection task. It can be used to be embedded in vehicle-side applications, improving the convenience and practicality of vehicle-side application deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of an application scenario of an end-to-end lane detection method in one embodiment of the present application.

[0041] Figure 2 This is a schematic diagram of the framework of a terminal in one embodiment of the present application.

[0042] Figure 3 Schematic diagram of the process of an end-to-end lane detection method in one embodiment of the present application.

[0043] Figure 4 This is a module diagram of an end-to-end lane detection system in one embodiment of the present application.

[0044] Figure 5 This is a schematic diagram of the internal structure of a device in another embodiment of the present application. DETAILED DESCRIPTION

[0045] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0046] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0047] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0049] Existing lane detection methods fall into at least two categories. The first involves: using a convolutional network A to extract features from the input front-view image on a grid basis; using a convolutional network B to process the lane feature map to obtain the position information and confidence level of each lane; and fitting the geometric position information of each lane point based on the coordinate information of each lane point. The second involves: obtaining an image containing annotated lane lines; converting the task into a lane instance segmentation task and performing end-to-end training; using a LaneNet network to perform pixel-by-pixel foreground and background classification, and using an embedding branch network to perform pixel embedding training to calculate pixels belonging to the same lane line; clustering the results from the previous step using the Mean-Shift algorithm to obtain the lane instance segmentation result; and finally, using HNet to predict the fitting of points on the same lane line and obtain the transformation matrix H that can convert the perspective into a bird's-eye view. The first approach uses two deep neural networks in series to calculate lane point information and confidence scores. This requires training the parameter weights of both networks sequentially, increasing the difficulty of network training. Furthermore, the relationship between the resulting lane point coordinates is unknown—that is, the set of points belonging to the same lane is unknown, making it difficult to fit the geometric information using traditional methods. The second approach uses instance segmentation to apply two network branches in parallel to the system. However, pixel embedding training itself is extremely difficult, requiring a significant increase in the number of samples required to train the lane segmentation model and requiring repeated experimentation with the embedding network parameters and loss functions. Both approaches use multiple deep neural networks to jointly predict lane information. The training and prediction process is cumbersome, and the chaining of multiple models leads to error accumulation, making it difficult to train a model with good performance. Furthermore, the high number of parameters requires more computing resources and consumes more power. For lane detection, traditional image algorithms are very poorly robust to environmental changes. While deep semantic segmentation models can achieve good detection results, they have a high number of parameters, making them unsuitable for deployment and computation on embedded devices.

[0050] In order to solve the above problems of the prior art, the solution of the present application is described in detail below with reference to specific embodiments.

[0051] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, smart cars, etc. The following describes exemplary applications of the device provided in the embodiments of the present application. The device provided in the embodiments of the present application can be implemented as various types of user terminals such as smart phones, smart watches, laptops, tablet computers, desktop computers, set-top boxes, mobile devices (for example, mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), intelligent voice interaction devices, smart home appliances, and vehicle-mounted terminals, and can also be implemented as servers. Below, an exemplary application when the device is implemented as a server will be described.

[0052] In one embodiment, one or more image sensing devices may be installed on the vehicle body, and the image sensing device may include devices such as cameras. For example, multiple cameras may be installed around the vehicle, each camera is responsible for a shooting angle around the vehicle, and the shooting angles of multiple cameras may cover the entire circumference of the vehicle. The camera on the vehicle body can be used to capture road images during the vehicle's driving process, and the road images can be transmitted to the server through the network for storage, and road images under different environments are obtained to construct a sample image set. The lane line detection model is trained based on the sample image set, and the trained lane line detection model is embedded in the vehicle-mounted system to process real-time road images collected by the vehicle side, which can be used for vehicle-side application navigation and assisted safe driving. The application scenario of the specific lane line can be adapted according to actual needs and is not limited here.

[0053] See also Figure 1 , Figure 1 The following is a schematic diagram of an application scenario of an end-to-end lane line detection system in one embodiment of the present application. The image acquisition device is usually installed on the vehicle body, and an image processing unit can also be provided to pre-process the images acquired by the image acquisition device, such as filtering out images with lower resolution, eliminating images that do not contain road surface information, etc. The specific image pre-processing can be set according to actual application requirements and is not limited here. The image processing unit can be installed on the vehicle body near the corresponding position of the image acquisition device to avoid data loss or data delay caused by long-distance data transmission. Of course, under the premise of ensuring the stability of the data transmission network 100, the image processing unit can also be set on the server 200. A communication connection is established between the image acquisition device and the image processing unit via a mobile network to complete the upload of sensor data. Constructing a sample data set and training a lane line detection model can be performed in the server 200.

[0054] In one embodiment, server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0055] In one embodiment, sample dataset construction and lane detection model training can also be performed on a terminal, which can be an in-vehicle terminal. After receiving real-time road images captured by the sensor acquisition device, the image processing unit preprocesses the real-time images and displays them in real time on the in-vehicle display terminal, allowing passengers in the vehicle to annotate lanes based on the displayed road images. Training samples corresponding to the sample images are obtained for training the lane detection model. In another embodiment, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, intelligent voice interaction device, smart home appliance, in-vehicle terminal, etc., but is not limited to these.

[0056] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the terminal 400 provided in an embodiment of the present application, Figure 2 The terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0057] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0058] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0059] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 410.

[0060] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0061] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0062] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0063] A network communication module 452 for reaching other computing devices via one or more (wired or wireless) network interfaces 420 , exemplary network interfaces 420 including Bluetooth, WiFi, and USB;

[0064] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0065] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0066] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 2 An end-to-end lane detection system 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: an image acquisition module 4551, a feature extraction module 4552, a lane parameter prediction module 4553 and a lane representation module 4554. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0067] The functions of each module will be described below.

[0068] In other embodiments, the system provided in the embodiments of the present application can be implemented in hardware. As an example, the system provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the end-to-end lane line detection method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0069] In some embodiments, a terminal or server can implement the end-to-end lane line detection method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a social application APP or a message sharing APP; it can also be a small program, that is, a program that can be run only by downloading it into a browser environment; it can also be a small program or web client program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0070] The end-to-end lane line detection method provided by the embodiment of the present application will be described below in combination with the exemplary application and implementation of the device provided by the embodiment of the present application.

[0071] See also Figure 3 , Figure 3 The figure is a flow chart of an end-to-end lane detection method in one embodiment of the present application. The end-to-end lane detection method provided by the present application includes the following steps.

[0072] Step S301: Acquire a front view road image of the vehicle.

[0073] In one embodiment, to better assist drivers in driving safely under various road conditions, an image acquisition device, such as a vehicle's front-facing camera, can capture images of the road ahead during driving. These images are referred to as forward-view road images. Lane markings are detected based on these images and displayed on an onboard terminal device for the driver's reference. The forward-view camera is used during normal driving to capture images in various scenarios, including those involving different road conditions, varying lighting conditions, urban areas, and elevated roads. This expands the distribution of training samples, making the model more robust and generalizing better.

[0074] Step S302: Input the forward road surface image into a feature extraction network to obtain a feature map.

[0075] In one embodiment, the network structure of the feature extraction network is as follows from left to right: a slicing module, a first convolution module, a first residual module, a second convolution module, and a second residual module;

[0076] The forward road image sequentially passes through the segmentation module, the first convolution module, the first residual module, the second convolution module, and the second residual module to obtain a corresponding feature map; wherein the first convolution module and the second convolution module have the same structure.

[0077] In one embodiment, the slicing module includes a slicing layer, a first feature splicing layer and a first convolutional layer. The forward road image passes through the slicing layer, the first feature splicing layer and the first convolutional layer in sequence to obtain a compressed feature map. Specifically, the forward road image can be first input into the slicing layer, and the forward road image can be sliced ​​to obtain multiple slices. These multiple slices are then input into the first feature splicing layer for concat splicing, and finally passed through the first convolutional layer for a Conv convolution operation. After this series of operations, the width and height of the feature map obtained are reduced to 1 / 2 of the original, which has the effect of expanding the receptive field and downsampling.

[0078] The first convolution module includes a second convolution layer, a first batch normalization layer, and a first activation function layer. The compressed feature map passes through the second convolution layer, the first batch normalization layer, and the first activation function layer in sequence to obtain the output of the first convolution module. Exemplarily, the first convolution module and the second convolution module are both composed of a series connection of Conv (convolution), BN (Batch Normalization), and Leaky Relu. In another embodiment, the activation function layer may also use a silu activation function.

[0079] The first residual module includes a first branch, a second branch, a second feature concatenation layer, a second batch normalization layer, a second activation function layer, a third convolution layer, a third batch normalization layer, and a third activation function layer. The first branch includes a convolution layer, and the second branch is composed of a fourth convolution layer, a fourth batch normalization layer, a fourth activation function layer, a residual network unit, and a fifth convolution layer in series. After the output of the first convolution module is processed by the first branch and the second branch, the feature maps output by the first and second branches are concatenated by the second feature concatenation layer. The feature maps are then processed sequentially by the second batch normalization layer, the second activation function layer, the third convolution layer, the third batch normalization layer, and the third activation function layer to obtain the output of the first residual module. In one embodiment, the first branch is a Conv, and the second branch is a branch consisting of a CBL, a res unit, and a Conv in series. The CBL includes a convolution layer, a batch normalization layer, and an activation function layer. After the feature maps are input into the first and second branches respectively, the two branches are concat-operated and sequentially subjected to BN (batch normalization), leaky relu, and CBL. The res unit is a basic residual network unit, that is, the input information is connected in series with two CBLs and then summed with the original input information.

[0080] The second residual module has a different number of residual network units from the first residual module, but the rest of the structure is the same. When there are multiple residual network units, each residual network unit is connected in series. For example, the second residual module may include three residual network units, and the network structure of a branch of the second residual module is composed of CBL, res unit, res unit, res unit, and Conv connected in series from left to right. The number of residual network units in the first residual module and the second residual module can be set and adjusted according to actual application requirements and is not limited here.

[0081] The above is the overall architecture of the feature extraction network. Through the feature extraction network, the feature map of the forward road image can be obtained for lane line prediction.

[0082] Step S303: Input the feature map into a pre-trained parameter prediction network to obtain lane line parameters of each lane line in the road surface image. The lane line parameters are used to represent the positional relationship between each point on the same lane line.

[0083] In one embodiment, before performing lane line prediction, the parameter prediction network needs to be pre-trained to optimize the network parameters of the parameter prediction network. The parameter prediction network is mainly used to predict the lane line parameters in the front road image. Assuming that the front road image contains four lanes, the lane line corresponding to each lane can be represented by a cubic curve. Specifically, the lane line can be expressed as x=ay^3+by^2+cy+d, where x and y represent the horizontal and vertical coordinates of the point corresponding to the lane line, respectively, and a, b, c, and d are the lane line parameters that the parameter prediction network needs to predict. The lane line parameters of different lane lines can be expressed as an, bn, cn, and dn (n={1,2,3,4}, representing four groups of lane line parameters from left to right, respectively).

[0084] In one embodiment, the pre-training step of the parameter prediction network includes:

[0085] Step S401: annotate lane lines in a preset sample image to obtain a training sample;

[0086] In one embodiment, the aforementioned road images under different vehicle driving environments may be collected in advance to construct a sample dataset, and then lane lines of each image in the sample dataset may be annotated to obtain training samples. Supervised learning is performed by annotating lane lines.

[0087] Step S402: Inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain parameter prediction results and lane line prediction results;

[0088] In one embodiment, to better perform supervised learning, an auxiliary supervision branch (i.e., the lane line prediction network branch) can be added during the model training phase. The feature map obtained by the feature extraction network is input into the parameter prediction network and the lane line prediction network branches, respectively, to calculate the parameter prediction results and lane line prediction results.

[0089] In one embodiment, the lane line prediction network branch includes at least three downsampling layers, a first fully connected layer and a loss function layer; the three downsampling layers each obtain a feature map of a different dimension, and the output of each downsampling layer is connected to the input of the first fully connected layer, and the feature maps of different dimensions are fused through the first fully connected layer, and then the loss value after feature fusion is calculated through the loss function layer. The downsampling layers can all be composed of convolution layers containing convolution kernels of preset sizes, and the sampling sizes of different downsampling layers are different. For example, feature maps of three scales of 76*76, 38*38 and 19*19 can be obtained respectively through the three downsampling layers. The feature maps of these three scales are then fused and classified through a fully connected layer, and the loss value of the feature map after classification by the fully connected layer is calculated through the loss function layer.

[0090] In one embodiment, the network structure of the parameter prediction network includes, from left to right, a sixth convolutional layer, a seventh convolutional layer, a feature compression layer, a second fully connected layer, and a third fully connected layer; the number of neurons in the third fully connected layer is equal to the number of lane line parameters to be predicted.

[0091] In one embodiment, the parameter prediction network uses two convolutional layers connected in series to perform a convolution operation. After the convolution operation is completed, the input feature compression layer performs a squeeze operation to compress the feature dimension. After the squeeze operation, two fully connected layers are connected in series. In one embodiment, the convolution kernel size is 3*3, the stride is 2, the pad is set to the same, and the number of neurons in the fully connected layer is set to 128 and 4, and 4 corresponds to the four lane line parameters to be regressed. The squeeze operation first compresses the features along the spatial dimension, converting each two-dimensional feature channel into a real number. This real number has a global receptive field to some extent, and the output dimension matches the number of input feature channels. It represents the global distribution of responses on the feature channels, and enables the layers close to the input to also obtain a global receptive field.

[0092] Step S403: determining a total network loss value based on the parameter prediction results and the lane line prediction results, and updating network parameters based on the total network loss value until the total loss value reaches a preset threshold, thereby completing pre-training of the parameter prediction network;

[0093] In one embodiment, determining a total network loss value based on the parameter prediction result and the lane line prediction result includes:

[0094] Performing lane line fitting according to the parameter prediction result to obtain a fitted lane line, wherein the parameter prediction result includes a first loss value of the parameter prediction network;

[0095] Using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch based on a similarity between the lane line prediction result and the label;

[0096] The first loss value and the second loss value are added together to obtain the total network loss value.

[0097] In one embodiment, the parameter prediction results obtained by the parameter prediction network may include lane parameters (an, bn, cn, and dn) and a first loss value of the current parameter prediction network. Based on the lane parameters and the aforementioned cubic curve representation, a corresponding lane line can be fitted, denoted as a fitted lane line. The fitted lane line serves as the label for the lane prediction network branch. The first loss value can be calculated using an absolute error loss function.

[0098] In one embodiment, the lane line prediction result may include a prediction frame of the lane line, which identifies the position of the lane line in the forward road image. The outer frame of the fitted lane line of the parameter prediction network is used as the true frame, and the intersection-and-union ratio between the predicted frame and the true frame is calculated to construct a loss function, wherein the loss function of the lane line prediction network branch can adopt ciou_loss, and the second loss value of the current lane line prediction network branch is calculated by the loss function, and the sum of the first loss value and the second loss value is used as the total loss value. Furthermore, based on the total loss value, the back propagation algorithm of the neural network can be used to optimize the parameters of the entire model until the total loss value is less than the set threshold, completing the training of the entire network.

[0099] Step S404: After completing the pre-training of the parameter prediction network, discard the lane line prediction network branch.

[0100] In one embodiment, after network training is completed, the lane line prediction network branch can be discarded and only the parameter prediction network can be retained. The parameter prediction network is used to perform forward propagation of lane line geometric parameter inference on the feature map of the real-time forward road image to obtain the lane line parameters of each lane line in the real-time forward road image.

[0101] Step S304: determining a geometric representation of each lane line in the front view road surface image according to the lane line parameters.

[0102] In one embodiment, after obtaining lane line parameters for the forward road image, lane line fitting is performed based on the lane line's geometric representation to obtain a lane line geometric representation for the real-time captured forward road image. This lane line geometric representation is displayed in real time on an in-vehicle terminal or other navigation display terminal to assist the driver in controlling the vehicle according to the fitted lane lines.

[0103] The fully end-to-end neural network proposed in this application boasts strong generalization capabilities and high detection accuracy. Compared to previously proposed end-to-end technologies, this application uses only a single network to predict lane geometry. Previous approaches either used multiple network models connected in series to predict lane point coordinates and then used traditional algorithms to fit the lane geometry, or used parallel and series methods to obtain lane geometry. These solutions were complex and difficult to implement, and did not meet the requirements for low parameter count and low power consumption required for embedded device deployment.

[0104] See also Figure 4 , Figure 4This is a module diagram of an end-to-end lane line detection system in one embodiment of the present application, which includes: an image acquisition module 4551, used to acquire a forward-view road surface image of a vehicle; a feature extraction module 4552, used to input the forward-view road surface image into a feature extraction network to obtain a feature map; a lane line parameter prediction module 4553, used to input the feature map into a pre-trained parameter prediction network to obtain lane line parameters for each lane line in the road surface image, wherein the lane line parameters are used to represent the positional relationship between each point on the same lane line; and a lane line representation module 4554, used to determine the geometric representation of each lane line in the forward-view road surface image based on the lane line parameters.

[0105] In one embodiment, the end-to-end lane line detection system also includes a pre-training module for pre-training a parameter prediction network. The pre-training steps of the parameter prediction network include: marking the lane lines in a preset sample image to obtain a training sample; inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain a parameter prediction result and a lane line prediction result; determining a total network loss value based on the parameter prediction result and the lane line prediction result, and updating the network parameters based on the total network loss value until the total loss value reaches a preset threshold, thereby completing the pre-training of the parameter prediction network; after completing the pre-training of the parameter prediction network, discarding the lane line prediction network branch.

[0106] In one embodiment, the pre-training module is also used to determine the total network loss value based on the parameter prediction results and the lane line prediction results, including: performing lane line fitting based on the parameter prediction results to obtain a fitted lane line, wherein the parameter prediction results include a first loss value of the parameter prediction network; using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch based on the similarity between the lane line prediction result and the label; adding the first loss value and the second loss value to obtain the total network loss value.

[0107] In one embodiment, the network structure of the feature extraction network is, from left to right, a slicing module, a first convolution module, a first residual module, a second convolution module, and a second residual module; the forward road image passes through the slicing module, the first convolution module, the residual module, the second convolution module, and the second residual module in sequence to obtain a corresponding feature map; wherein the first convolution module and the second convolution module have the same structure.

[0108] In one embodiment, the slicing module includes a slicing layer, a first feature splicing layer, and a first convolution layer. The forward road image passes through the slicing layer, the feature splicing layer, and the convolution layer in sequence to obtain a compressed feature map; the first convolution module includes a second convolution layer, a first batch normalization layer, and a first activation function layer. The compressed feature map passes through the second convolution layer, the first batch normalization layer, and the first activation function layer in sequence to obtain the output of the first convolution module; the first residual module includes a first branch, a second branch, a second feature splicing layer, a second batch normalization layer, a second activation function layer, a third convolution layer, a third batch normalization layer, and a third activation function layer. The first branch includes a convolution layer. The second branch is composed of a fourth convolutional layer, a fourth batch normalization layer, a fourth activation function layer, a residual network unit and a fifth convolutional layer connected in series; the output of the first convolution module is processed by the first branch and the second branch respectively, and the feature maps output by the first branch and the second branch are spliced ​​through the second feature splicing layer, and then processed in sequence by the second batch normalization layer, the second activation function layer, the third convolutional layer, the third batch normalization layer and the third activation function layer to obtain the output of the first residual module; the second residual module has a different number of residual network units from the first residual module, and the rest of the structure is the same. When there are multiple residual network units, each residual network unit is connected in series in sequence.

[0109] In one embodiment, the lane line prediction network branch includes at least: three downsampling layers, a first fully connected layer and a loss function layer; the three downsampling layers each obtain a feature map of a different dimension, the output of each downsampling layer is connected to the input of the first fully connected layer, the feature maps of different dimensions are fused through the first fully connected layer, and then the loss value after feature fusion is calculated through the loss function layer.

[0110] In one embodiment, the network structure of the parameter prediction network includes, from left to right, a sixth convolutional layer, a seventh convolutional layer, a feature compression layer, a second fully connected layer, and a third fully connected layer; the number of neurons in the third fully connected layer is equal to the number of lane line parameters to be predicted.

[0111] In one embodiment, the convolution kernels of the sixth convolution layer and the seventh convolution layer are 3*3 convolution kernels, and the moving step length of the convolution kernel is 2.

[0112] In one embodiment, the geometric representation of the lane line is represented by a cubic curve.

[0113] The above end-to-end lane detection system can be implemented in the form of a computer program. Figure 5The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0114] Each module in the above-mentioned end-to-end lane detection system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the terminal's memory in hardware form, or stored in the terminal's memory in software form, so that a processor can call and execute the corresponding operations of each module. The processor can be a central processing unit (CPU), microprocessor, single-chip microcomputer, etc.

[0115] like Figure 5 Figure 2 is a schematic diagram of the internal structure of a computer device in one embodiment. A computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed: acquiring a forward-view road image of a vehicle; inputting the forward-view road image into a feature extraction network to obtain a feature map; inputting the feature map into a pre-trained parameter prediction network to obtain lane line parameters for each lane line in the road image. The lane line parameters are used to represent the positional relationship between points on the same lane line; and determining a geometric representation of each lane line in the forward-view road image based on the lane line parameters.

[0116] In one embodiment, when the above-mentioned processor is executed, the pre-training step of the parameter prediction network implemented includes: marking the lane lines in the preset sample image to obtain a training sample; inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain a parameter prediction result and a lane line prediction result; determining the total network loss value based on the parameter prediction result and the lane line prediction result, and updating the network parameters based on the total network loss value until the total loss value reaches a preset threshold, thereby completing the pre-training of the parameter prediction network; after completing the pre-training of the parameter prediction network, discarding the lane line prediction network branch.

[0117] In one embodiment, when the above-mentioned processor is executed, the total network loss value is determined based on the parameter prediction result and the lane line prediction result, including: performing lane line fitting based on the parameter prediction result to obtain a fitted lane line, wherein the parameter prediction result includes a first loss value of the parameter prediction network; using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch based on the similarity between the lane line prediction result and the label; and adding the first loss value and the second loss value to obtain the total network loss value.

[0118] In one embodiment, when the above-mentioned processor is executed, the network structure of the feature extraction network implemented is, from left to right,: a slicing module, a first convolution module, a first residual module, a second convolution module, and a second residual module; the front road image passes through the slicing module, the first convolution module, the residual module, the second convolution module, and the second residual module in sequence to obtain a corresponding feature map; wherein the structures of the first convolution module and the second convolution module are the same.

[0119] In one embodiment, when the above-mentioned processor is executed, the implemented slicing module includes a slicing layer, a first feature splicing layer and a first convolution layer, and the front road image passes through the slicing layer, the feature splicing layer and the convolution layer in sequence to obtain a compressed feature map; the first convolution module includes a second convolution layer, a first batch normalization layer and a first activation function layer, and the compressed feature map passes through the second convolution layer, the first batch normalization layer and the first activation function layer in sequence to obtain the output of the first convolution module; the first residual module includes a first branch, a second branch, a second feature splicing layer, a second batch normalization layer, a second activation function layer, a third convolution layer, a third batch normalization layer and a third activation function layer, and the first branch It includes a convolution layer, and the second branch is composed of a fourth convolution layer, a fourth batch normalization layer, a fourth activation function layer, a residual network unit and a fifth convolution layer in series; the output of the first convolution module is processed by the first branch and the second branch respectively, and the feature maps output by the first branch and the second branch are spliced ​​through the second feature splicing layer, and then processed in sequence by the second batch normalization layer, the second activation function layer, the third convolution layer, the third batch normalization layer and the third activation function layer to obtain the output of the first residual module; the second residual module has a different number of residual network units from the first residual module, and the rest of the structure is the same. When there are multiple residual network units, each residual network unit is connected in series in sequence.

[0120] In one embodiment, when the above-mentioned processor is executed, the lane line prediction network branch implemented includes at least: three downsampling layers, a first fully connected layer and a loss function layer; the three downsampling layers each obtain a feature map of a different dimension, the output of each downsampling layer is connected to the input of the first fully connected layer, the feature maps of different dimensions are fused through the first fully connected layer, and then the loss value after feature fusion is calculated through the loss function layer.

[0121] In one embodiment, when the above-mentioned processor is executed, the network structure of the parameter prediction network implemented includes, from left to right, the sixth convolutional layer, the seventh convolutional layer, the feature compression layer, the second fully connected layer and the third fully connected layer; the number of neurons in the third fully connected layer is equal to the number of lane line parameters to be predicted.

[0122] In one embodiment, when the above-mentioned processor is executed, the convolution kernels of the sixth convolution layer and the seventh convolution layer implemented are 3*3 convolution kernels, and the moving step size of the convolution kernel is 2.

[0123] In one embodiment, when the above processor is executed, the geometric representation of the lane line implemented is represented by a cubic curve.

[0124] In one embodiment, the above-mentioned computer device can be used as a server, including but not limited to an independent physical server, or a server cluster composed of multiple physical servers. The computer device can also be used as a terminal, including but not limited to a mobile phone, tablet computer, personal digital assistant or smart device. Figure 5 As shown, the computer device includes a processor, a non-volatile storage medium, an internal memory, a display screen, and a network interface connected via a system bus.

[0125] Among them, the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The non-volatile storage medium of the computer device stores an operating system and a computer program. The computer program can be executed by the processor to implement a fault diagnosis method provided in each of the above embodiments. The internal memory in the computer device provides a cached operating environment for the operating system and computer program in the non-volatile storage medium. The display interface can display data through a display screen. The display screen can be a touch screen, such as a capacitive screen or an electronic screen, which can generate corresponding instructions by receiving a click operation on the control displayed on the touch screen.

[0126] Those skilled in the art will understand that Figure 5 The structure of the computer device shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0127] In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which, when executed by a processor, implements the following steps: acquiring a forward-view road surface image of a vehicle; inputting the forward-view road surface image into a feature extraction network to obtain a feature map; inputting the feature map into a pre-trained parameter prediction network to obtain lane line parameters for each lane line in the road surface image, wherein the lane line parameters are used to represent the positional relationship between points on the same lane line; and determining a geometric representation of each lane line in the forward-view road surface image based on the lane line parameters.

[0128] In one embodiment, when the computer program is executed by a processor, the pre-training step of the parameter prediction network implemented includes: marking the lane lines in a preset sample image to obtain a training sample; inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain a parameter prediction result and a lane line prediction result; determining the total network loss value based on the parameter prediction result and the lane line prediction result, and updating the network parameters based on the total network loss value until the total loss value reaches a preset threshold, thereby completing the pre-training of the parameter prediction network; after completing the pre-training of the parameter prediction network, discarding the lane line prediction network branch.

[0129] In one embodiment, when the computer program is executed by a processor, the method implemented to determine the total network loss value based on the parameter prediction results and the lane line prediction results includes: performing lane line fitting based on the parameter prediction results to obtain a fitted lane line, wherein the parameter prediction results include a first loss value of the parameter prediction network; using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch based on the similarity between the lane line prediction result and the label; and adding the first loss value to the second loss value to obtain the total network loss value.

[0130] In one embodiment, when the computer program is executed by a processor, the network structure of the feature extraction network implemented is, from left to right,: a slicing module, a first convolution module, a first residual module, a second convolution module, and a second residual module; the forward road image passes through the slicing module, the first convolution module, the residual module, the second convolution module, and the second residual module in sequence to obtain a corresponding feature map; wherein the first convolution module and the second convolution module have the same structure.

[0131] In one embodiment, when the computer program is executed by a processor, the implemented slicing module includes a slicing layer, a first feature splicing layer, and a first convolution layer, and the forward road image passes through the slicing layer, the feature splicing layer, and the convolution layer in sequence to obtain a compressed feature map; the first convolution module includes a second convolution layer, a first batch normalization layer, and a first activation function layer, and the compressed feature map passes through the second convolution layer, the first batch normalization layer, and the first activation function layer in sequence to obtain the output of the first convolution module; the first residual module includes a first branch, a second branch, a second feature splicing layer, a second batch normalization layer, a second activation function layer, a third convolution layer, a third batch normalization layer, and a third activation function layer, and the first One branch includes a convolution layer, and the second branch is composed of a fourth convolution layer, a fourth batch normalization layer, a fourth activation function layer, a residual network unit and a fifth convolution layer connected in series; the output of the first convolution module is processed by the first branch and the second branch respectively, and the feature maps output by the first branch and the second branch are spliced ​​through the second feature splicing layer, and then processed in sequence by the second batch normalization layer, the second activation function layer, the third convolution layer, the third batch normalization layer and the third activation function layer to obtain the output of the first residual module; the second residual module has a different number of residual network units from the first residual module, and the rest of the structure is the same. When there are multiple residual network units, each residual network unit is connected in series in sequence.

[0132] In one embodiment, when the computer program is executed by a processor, the implemented lane line prediction network branch includes at least: three downsampling layers, a first fully connected layer and a loss function layer; the three downsampling layers each obtain a feature map of a different dimension, the output of each downsampling layer is connected to the input of the first fully connected layer, the feature maps of different dimensions are fused through the first fully connected layer, and then the loss value after feature fusion is calculated through the loss function layer.

[0133] In one embodiment, when the instruction is executed by the processor, the network structure of the parameter prediction network implemented includes, from left to right, the sixth convolutional layer, the seventh convolutional layer, the feature compression layer, the second fully connected layer and the third fully connected layer; the number of neurons in the third fully connected layer is equal to the number of lane line parameters to be predicted.

[0134] In one embodiment, when the instruction is executed by the processor, the convolution kernels of the sixth convolution layer and the seventh convolution layer implemented adopt 3*3 convolution kernels, and the moving step size of the convolution kernel is 2.

[0135] In one embodiment, when the instruction is executed by the processor, the geometric representation of the lane line is implemented using a cubic curve.

[0136] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or the like.

[0137] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. An end-to-end lane detection method, characterized in that: include: Acquire a forward road image of the vehicle; Inputting the forward road surface image into a feature extraction network to obtain a feature map; Inputting the feature map into a pre-trained parameter prediction network to obtain lane line parameters of each lane line in the road surface image, wherein the lane line parameters are used to represent the positional relationship between each point on the same lane line; The pre-training step of the parameter prediction network includes: marking the lane lines in the preset sample image to obtain a training sample; inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain a parameter prediction result and a lane line prediction result; determining the total network loss value according to the parameter prediction result and the lane line prediction result, and updating the network parameters according to the total network loss value until the total loss value reaches a preset threshold, completing the pre-training of the parameter prediction network; after completing the pre-training of the parameter prediction network, discarding the lane line prediction network branch; determining the total network loss value according to the parameter prediction result and the lane line prediction result, including: performing lane line fitting according to the parameter prediction result to obtain a fitted lane line, wherein the parameter prediction result includes a first loss value of the parameter prediction network; using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch according to the similarity between the lane line prediction result and the label; adding the first loss value and the second loss value to obtain the total network loss value; A geometric representation of each lane line in the front-view road surface image is determined according to the lane line parameters.

2. The end-to-end lane detection method according to claim 1, characterized in that: The network structure of the feature extraction network is as follows from left to right: slicing module, first convolution module, first residual module, second convolution module and second residual module; The forward road image sequentially passes through the segmentation module, the first convolution module, the first residual module, the second convolution module, and the second residual module to obtain a corresponding feature map; wherein the first convolution module and the second convolution module have the same structure.

3. The end-to-end lane detection method according to claim 2, characterized in that: The slicing module includes a slicing layer, a first feature splicing layer, and a first convolution layer, and the front view road image passes through the slicing layer, the first feature splicing layer, and the first convolution layer in sequence to obtain a compressed feature map; The first convolution module includes a second convolution layer, a first batch normalization layer, and a first activation function layer, and the compressed feature map passes through the second convolution layer, the first batch normalization layer, and the first activation function layer in sequence to obtain the output of the first convolution module; The first residual module includes a first branch, a second branch, a second feature splicing layer, a second batch normalization layer, a second activation function layer, a third convolution layer, a third batch normalization layer and a third activation function layer, the first branch includes a convolution layer, and the second branch is composed of a fourth convolution layer, a fourth batch normalization layer, a fourth activation function layer, a residual network unit and a fifth convolution layer connected in series; the output of the first convolution module is processed by the first branch and the second branch respectively, and the feature maps output by the first branch and the second branch are spliced ​​through the second feature splicing layer, and then processed in sequence by the second batch normalization layer, the second activation function layer, the third convolution layer, the third batch normalization layer and the third activation function layer to obtain the output of the first residual module; The second residual module has a different number of residual network units from the first residual module, but the rest of the structure is the same. When there are multiple residual network units, each residual network unit is connected in series in sequence.

4. The end-to-end lane detection method according to claim 1, characterized in that: The lane line prediction network branch includes at least: three downsampling layers, a first fully connected layer, and a loss function layer; the three downsampling layers each obtain a feature map of a different dimension, the output of each downsampling layer is connected to the input of the first fully connected layer, the feature maps of different dimensions are fused through the first fully connected layer, and the loss value after feature fusion is calculated through the loss function layer.

5. The end-to-end lane detection method according to claim 1, characterized in that: The network structure of the parameter prediction network includes, from left to right, the sixth convolutional layer, the seventh convolutional layer, the feature compression layer, the second fully connected layer, and the third fully connected layer; The number of neurons in the third fully connected layer is equal to the number of lane line parameters to be predicted.

6. The end-to-end lane detection method according to claim 5, characterized in that: The convolution kernels of the sixth convolution layer and the seventh convolution layer adopt 3*3 convolution kernels, and the moving step length of the convolution kernel is 2.

7. The end-to-end lane detection method according to any one of claims 1 to 6, characterized in that: The geometric representation of the lane line is represented by a cubic curve.

8. An end-to-end lane detection system, characterized in that: include: An image acquisition module, used to acquire a front view road image of the vehicle; A feature extraction module, configured to input the forward road surface image into a feature extraction network to obtain a feature map; a lane line parameter prediction module, configured to input the feature map into a pre-trained parameter prediction network to obtain lane line parameters for each lane line in the road surface image, wherein the lane line parameters are used to represent the positional relationship between points on the same lane line; The pre-training step of the parameter prediction network includes: marking the lane lines in the preset sample image to obtain a training sample; inputting the feature map of the training sample into the parameter prediction network and the preset lane line prediction network branch respectively to obtain a parameter prediction result and a lane line prediction result; determining the total network loss value according to the parameter prediction result and the lane line prediction result, and updating the network parameters according to the total network loss value until the total loss value reaches a preset threshold, completing the pre-training of the parameter prediction network; after completing the pre-training of the parameter prediction network, discarding the lane line prediction network branch; determining the total network loss value according to the parameter prediction result and the lane line prediction result, including: performing lane line fitting according to the parameter prediction result to obtain a fitted lane line, wherein the parameter prediction result includes a first loss value of the parameter prediction network; using the fitted lane line as a label, and obtaining a second loss value of the lane line prediction network branch according to the similarity between the lane line prediction result and the label; adding the first loss value and the second loss value to obtain the total network loss value; A lane line representation module is used to determine the geometric representation of each lane line in the front view road surface image according to the lane line parameters.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the end-to-end lane detection method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the end-to-end lane detection method according to any one of claims 1 to 7 are implemented.

Images