Three-dimensional lane line identification method and device, storage medium and electronic equipment
Through the three-dimensional lane line recognition method, the lane line recognition is used to use a pre-trained model to solve the error problem of two-dimensional lane line perception on the concave and convex road surface, improve the perception accuracy, and avoid the occurrence of vehicle accidents.
Patent Information
- Application Number
- CN202310802323.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-30
- Publication Date
- 2024-12-31
AI Technical Summary
The two-dimensional lane line perception technology has a large error in perception results when uneven roads, which can easily lead to vehicle driving accidents.
Using the three-dimensional lane line recognition method, by obtaining the vehicle's forward-view image and inputting a pre-trained three-dimensional lane line recognition model, the backbone network, spatial transformation network and encoder decoder are used to perform two-dimensional feature encoding and three-dimensional feature decoding to generate closer Information about real three-dimensional lane lines.
The accuracy of perceived results in uneven roads is improved, and the occurrence of vehicle driving accidents is reduced.
Smart Images
Figure CN119229395A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automobile technology, and in particular to a three-dimensional lane line recognition method and a device, a storage medium, and an electronic device thereof. Background Art
[0002] With the development of society and urbanization, the social functions and spatial structures of cities are becoming increasingly complex. Among them, the complexity of the road traffic network is gradually increasing the learning cost and driving mental strength required for people to drive. In this context, the autonomous driving technology of automobiles aims to liberate any human active operation and automatically and safely complete travel tasks by relying on the collaborative cooperation of artificial intelligence, visual computing, radar, monitoring devices and global positioning systems. The perception task is equivalent to the "eyes" of the car in the autonomous driving technology of automobiles. It is the basic work in autonomous driving. On this basis, the autonomous driving car combines the road perception results with other task modules to achieve the final motion control.
[0003] Among all road perception targets, lane lines are the most critical ones. The current mainstream lane line perception technology is based on two-dimensional lane line perception, which usually consists of three steps: (1) segmenting the lane lines of the two-dimensional image and extracting the pixel set belonging to the lane lines in the image; (2) transforming the lane line segmentation results from the image coordinate system to the BEV coordinate system through spatial transformation; (3) by strongly assuming that the world is a plane, the two-dimensional lane line curve fitting results are used to approximately represent the real three-dimensional world. However, this assumption itself is unreasonable. This method has large perception errors on uneven roads, which can easily lead to vehicle driving accidents. Summary of the invention
[0004] The purpose of this application is to propose a three-dimensional lane line recognition method and its device, storage medium, and electronic device to solve the technical problem that the perception result error of the two-dimensional lane line perception implementation scheme is large when the road surface is uneven, which may easily lead to vehicle driving accidents.
[0005] To achieve the above objectives, the present application provides a three-dimensional lane line recognition method, including:
[0006] Acquire a front view image of the vehicle;
[0007] Inputting the vehicle front view image into a pre-trained three-dimensional lane line recognition model to perform lane line recognition to obtain three-dimensional lane line information;
[0008] Among them, the three-dimensional lane line recognition model includes a backbone network, a spatial transformation network, an encoder, and a decoder connected in sequence. The backbone network is used to receive a vehicle front view image, and perform two-dimensional feature encoding on the vehicle front view image to obtain a two-dimensional lane line segmentation result; the spatial transformation network is used to convert the two-dimensional lane line segmentation result to a top-view coordinate system; the encoder is used to perform three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system to obtain three-dimensional lane line features; the decoder is used to decode the three-dimensional lane line features to obtain three-dimensional lane line information.
[0009] In some schemes, the training of the three-dimensional lane line recognition model includes a first stage and a second stage. The first stage includes obtaining a backbone network to be trained, and pre-training the backbone network to be trained using vehicle front view image samples and their corresponding two-dimensional annotations to obtain a pre-trained backbone network; the second stage includes constructing a three-dimensional lane line recognition model to be trained based on the pre-trained backbone network and the spatial transformation network, encoder, and decoder to be trained, and training the three-dimensional lane line recognition model to be trained using vehicle front view image samples and their corresponding three-dimensional annotations to obtain a three-dimensional lane line recognition model.
[0010] In some embodiments, the two-dimensional annotation is a mask image corresponding to the vehicle front view image sample.
[0011] In some embodiments, the three-dimensional annotation is three-dimensional position information of lane lines in the vehicle front view image sample.
[0012] In some schemes, the first stage specifically includes: using the two-dimensional annotation as a supervision signal of the backbone network to be trained to train the backbone network to obtain a pre-trained backbone network;
[0013] The second stage specifically includes:
[0014] The encoder and decoder are trained using the three-dimensional annotation as supervisory signals of the encoder and decoder.
[0015] In some embodiments, the second stage specifically includes:
[0016] According to the internal and external parameters of the vehicle's front-view camera, the three-dimensional position information of the lane line in the vehicle's front-view image sample is projected into the two-dimensional space under the viewing angle of the vehicle's front-view camera to generate a two-dimensional annotation, and the pre-trained backbone network is trained using the two-dimensional annotation as a supervision signal of the pre-trained backbone network.
[0017] The present application also provides a three-dimensional lane line recognition device, including:
[0018] An image acquisition module, used for acquiring a front view image of a vehicle;
[0019] An image recognition module is used to input the vehicle front view image into a pre-trained three-dimensional lane line recognition model to perform lane line recognition and obtain three-dimensional lane line information;
[0020] Among them, the three-dimensional lane line recognition model includes a backbone network, a spatial transformation network, an encoder, and a decoder connected in sequence. The backbone network is used to receive a vehicle front view image, and perform two-dimensional feature encoding on the vehicle front view image to obtain a two-dimensional lane line segmentation result; the spatial transformation network is used to convert the two-dimensional lane line segmentation result to a top-view coordinate system; the encoder is used to perform three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system to obtain three-dimensional lane line features; the decoder is used to decode the three-dimensional lane line features to obtain three-dimensional lane line information.
[0021] In some schemes, the training of the three-dimensional lane line recognition model includes a first stage and a second stage. The first stage includes obtaining a backbone network to be trained, and pre-training the backbone network to be trained using vehicle front view image samples and their corresponding two-dimensional annotations to obtain a pre-trained backbone network; the second stage includes constructing a three-dimensional lane line recognition model to be trained based on the pre-trained backbone network and the spatial transformation network, encoder, and decoder to be trained, and training the three-dimensional lane line recognition model to be trained using vehicle front view image samples and their corresponding three-dimensional annotations to obtain a three-dimensional lane line recognition model.
[0022] In some embodiments, the two-dimensional annotation is a mask image corresponding to the vehicle front view image sample, and the three-dimensional annotation is the three-dimensional position information of the lane line in the vehicle front view image sample.
[0023] In some schemes, the first stage specifically includes: using the two-dimensional annotation as a supervision signal of the backbone network to be trained to train the backbone network to obtain a pre-trained backbone network;
[0024] The second stage specifically includes:
[0025] Training the encoder and decoder using the three-dimensional annotation as supervisory signals of the encoder and decoder;
[0026] According to the internal and external parameters of the vehicle's front-view camera, the three-dimensional position information of the lane line in the vehicle's front-view image sample is projected into the two-dimensional space under the viewing angle of the vehicle's front-view camera to generate a two-dimensional annotation, and the pre-trained backbone network is trained using the two-dimensional annotation as a supervision signal of the pre-trained backbone network.
[0027] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the three-dimensional lane line recognition method as described above is implemented.
[0028] An embodiment of the present application also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the three-dimensional lane line recognition method as described above when executing the computer program.
[0029] The embodiment of the present application provides a three-dimensional lane line recognition method and its device, storage medium, and electronic device, which realizes the recognition of lane lines in a vehicle front view image based on a three-dimensional lane line recognition model. The three-dimensional lane line recognition model is connected to a backbone network, a spatial transformation network, an encoder, and a decoder in sequence. The vehicle front view image is received through the backbone network, and the vehicle front view image is encoded with two-dimensional features to obtain a two-dimensional lane line segmentation result, and the spatial transformation network is used to convert the two-dimensional lane line segmentation result to a top-view coordinate system. Compared with the method in the prior art that strongly assumes that the world is a plane and uses a two-dimensional lane line curve fitting result to approximately represent the real three-dimensional world, this embodiment uses an encoder to encode the lane segmentation result in the top-view coordinate system to obtain a three-dimensional lane line feature, and uses a decoder to decode the three-dimensional lane line feature to obtain three-dimensional lane line information. The obtained three-dimensional lane line information can be closer to the real three-dimensional information of the lane line. Therefore, it can effectively improve the accuracy of the perception result on uneven roads and avoid the occurrence of vehicle driving accidents. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0031] Figure 1 This is a flowchart of a three-dimensional lane recognition method in one embodiment of the present application.
[0032] Figure 2 This is a diagram showing the training principle of a three-dimensional lane recognition model in one embodiment of the present application.
[0033] Figure 3 This is a structural framework diagram of a three-dimensional lane line recognition device in one embodiment of the present application. DETAILED DESCRIPTION
[0034] The detailed description of the drawings is intended as an illustration of the current embodiment of the present application, and is not intended to represent the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included in the spirit and scope of the present application.
[0035] See also Figure 1 An embodiment of the present application provides a three-dimensional lane line recognition method, comprising the following steps:
[0036] Step S1, obtaining a front view image of a vehicle;
[0037] Step S2: input the vehicle front view image into a pre-trained three-dimensional lane line recognition model to perform lane line recognition to obtain three-dimensional lane line information.
[0038] The three-dimensional lane line recognition model includes a backbone network, a spatial transformation network, an encoder, and a decoder connected in sequence;
[0039] The backbone network is used to receive a vehicle front view image, and perform two-dimensional feature encoding on the vehicle front view image to obtain a two-dimensional lane line segmentation result; specifically, the backbone network can use a convolutional neural network (CNN) to extract two-dimensional features; the selection of the backbone network mainly depends on the application scenario, such as a preference for efficiency, a preference for accuracy, or a relative balance between efficiency and accuracy; the backbone network is placed at the bottom layer of the overall network model, and is responsible for encoding and identifying the geometric information of the vehicle front view image in the two-dimensional space, and outputs a two-dimensional lane segmentation result on this basis;
[0040] The spatial transformation network is used to transform the two-dimensional lane segmentation result into a top-view coordinate system using IPM (Inverse Perspective Mapping); specifically, the spatial transformation network performs perspective transformation on the lane segmentation result, maps the lane segmentation result from a pixel coordinate system to a physical coordinate system through a transformation matrix, and performs perspective transformation to transform the image into a top-view perspective; wherein, the internal pixel values of the lane segmentation result are normalized and balanced before the spatial transformation;
[0041] The encoder is used to perform three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system to obtain three-dimensional lane line features; specifically, the lane line segmentation result in the top-view coordinate system is converted into a three-dimensional lane line. The three-dimensional lane line can be obtained by projecting the lane line on the top-view upward, that is, scaling it in the vertical direction. Furthermore, a suitable feature representation method can be selected. The three-dimensional lane line can be represented as a series of point clouds or line segments, and feature encoding can be performed on each point or line segment.
[0042] The decoder is used to decode the three-dimensional lane line features to obtain three-dimensional lane line information; specifically, a suitable decoding method can be selected to decode the result output by the encoder. For feature encoding of point clouds or line segments, commonly used decoding methods include inverse image transformation, random forest, etc.
[0043] The method of this embodiment receives a vehicle front view image through a backbone network, performs two-dimensional feature encoding on the vehicle front view image to obtain a two-dimensional lane line segmentation result, and uses a spatial transformation network to convert the two-dimensional lane line segmentation result to a top-view coordinate system; compared with the prior art method of using a two-dimensional lane line curve fitting result to approximately represent the real three-dimensional world by strongly assuming that the world is a plane, this embodiment performs three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system through an encoder to obtain a three-dimensional lane line feature, and decodes the three-dimensional lane line feature through a decoder to obtain three-dimensional lane line information. The obtained three-dimensional lane line information can be closer to the real three-dimensional information of the lane line. Therefore, it can effectively improve the accuracy of perception results on uneven roads and avoid the occurrence of vehicle driving accidents.
[0044] In some embodiments, see Figure 2 , the training of the three-dimensional lane line recognition model includes a first stage and a second stage;
[0045] The first stage includes obtaining a backbone network to be trained, and pre-training the backbone network to be trained using vehicle front view image samples and their corresponding two-dimensional annotations to obtain a pre-trained backbone network; specifically, in the first stage, this embodiment expands the lane line segmentation task based on two-dimensional images to be a separate basic upstream task, and pre-trains the image segmentation sub-network using a large number of vehicle front view image samples and their two-dimensional lane line annotation data, rather than being limited to using specific data in specific downstream scenarios for training;
[0046] The second stage includes constructing a three-dimensional lane line recognition model to be trained based on the pre-trained backbone network and the spatial transformation network, encoder, and decoder to be trained, and using vehicle front view image samples and their corresponding three-dimensional annotations to train the three-dimensional lane line recognition model to obtain a three-dimensional lane line recognition model; specifically, in the second stage, according to the specific downstream scenario, a pre-trained image segmentation sub-network model is used as the backbone network to be trained, and on this basis, a relatively small amount of vehicle front view image samples and their three-dimensional lane line annotation data are used to train the encoder and decoder parts of the three-dimensional lane line recognition model, and the backbone network to be trained is fine-tuned; the backbone network weights that have been pre-trained in the upstream task can be reused in different downstream tasks, which can reduce the perception model's demand for three-dimensional annotations while improving the perception performance of the perception model.
[0047] In some embodiments, the two-dimensional annotation is a mask image corresponding to the vehicle front view image sample.
[0048] Specifically, when performing two-dimensional annotation, it is necessary to manually create a mask image and annotate the lane lines on the vehicle front view image sample to obtain the corresponding mask image. Specifically, according to the requirements of the actual data set, a blank mask image with the same resolution as the vehicle front view image sample can be created, or the vehicle front view image sample can be directly edited to create a new transparent mask layer. As a supervisory signal, the mask image has better target edge information, stronger anti-occlusion and blocking capabilities, and better robustness and generalization capabilities.
[0049] In some embodiments, the three-dimensional annotation is the three-dimensional position information of the lane line in the vehicle front view image sample, which is composed of a series of three-dimensional points.
[0050] In some embodiments, the first stage specifically includes: using the two-dimensional annotation as a supervisory signal of the backbone network to be trained to train the backbone network to obtain a pre-trained backbone network;
[0051] The second stage specifically includes: using the three-dimensional annotation as the supervisory signal of the encoder and decoder to train the encoder and decoder to generate three-dimensional lane line information based on the two-dimensional lane line.
[0052] Specifically, the supervisory signal of the network model is used to measure the difference between the model's predicted results and the actual results, and is a key indicator for optimizing the training of the network model; Figure 2 As shown, the encoder has three layers, each of which is composed of a maximum pooling layer and a convolutional layer; the encoder can learn the coordinate information of the three-dimensional lane in the two-dimensional mask, including the horizontal coordinate X to the right, the vertical coordinate Y to the front, and the height coordinate Z to the top, but this three-dimensional coordinate information is an abstract state after feature encoding, and the decoder needs to be parsed to output the final three-dimensional result.
[0053] In some embodiments, the second stage further includes:
[0054] According to the internal and external parameters of the vehicle's front-view camera, the three-dimensional position information of the lane line in the vehicle's front-view image sample is projected into the two-dimensional space under the viewing angle of the vehicle's front-view camera to generate a two-dimensional annotation, and the pre-trained backbone network is trained using the two-dimensional annotation as a supervision signal of the pre-trained backbone network.
[0055] Specifically, in response to the limitations brought by the fixed image segmentation sub-network model parameters in the second stage, this embodiment, based on the above-proposed training method, still supervises the pre-trained backbone network with two-dimensional lane line segmentation annotation when training the encoder and decoder in a specific downstream scenario, thereby freeing up the model parameters of the backbone network for collaborative fine-tuning, which is beneficial to the information interaction within the model.
[0056] See also Figure 3 Another embodiment of the present application further provides a three-dimensional lane line recognition device, comprising:
[0057] An image acquisition module 1 is used to acquire a front view image of a vehicle;
[0058] Image recognition module 2, used for inputting the vehicle front view image into a pre-trained three-dimensional lane line recognition model to perform lane line recognition and obtain three-dimensional lane line information;
[0059] Among them, the three-dimensional lane line recognition model includes a backbone network, a spatial transformation network, an encoder, and a decoder connected in sequence. The backbone network is used to receive a vehicle front view image, and perform two-dimensional feature encoding on the vehicle front view image to obtain a two-dimensional lane line segmentation result; the spatial transformation network is used to convert the two-dimensional lane line segmentation result to a top-view coordinate system; the encoder is used to perform three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system to obtain three-dimensional lane line features; the decoder is used to decode the three-dimensional lane line features to obtain three-dimensional lane line information.
[0060] In some embodiments, the training of the three-dimensional lane line recognition model includes a first stage and a second stage. The first stage includes obtaining a backbone network to be trained, and pre-training the backbone network to be trained using vehicle front view image samples and their corresponding two-dimensional annotations to obtain a pre-trained backbone network; the second stage includes constructing a three-dimensional lane line recognition model to be trained based on the pre-trained backbone network and the spatial transformation network, encoder, and decoder to be trained, and training the three-dimensional lane line recognition model to be trained using vehicle front view image samples and their corresponding three-dimensional annotations to obtain a three-dimensional lane line recognition model.
[0061] In some embodiments, the two-dimensional annotation is a mask image corresponding to the vehicle front view image sample, and the three-dimensional annotation is three-dimensional position information of the lane line in the vehicle front view image sample.
[0062] In some embodiments, the first stage specifically includes: using the two-dimensional annotation as a supervisory signal of the backbone network to be trained to train the backbone network to obtain a pre-trained backbone network;
[0063] The second stage specifically includes:
[0064] Training the encoder and decoder using the three-dimensional annotation as supervisory signals of the encoder and decoder;
[0065] According to the internal and external parameters of the vehicle's front-view camera, the three-dimensional position information of the lane line in the vehicle's front-view image sample is projected into the two-dimensional space under the viewing angle of the vehicle's front-view camera to generate a two-dimensional annotation, and the pre-trained backbone network is trained using the two-dimensional annotation as a supervision signal of the pre-trained backbone network.
[0066] It should be noted that the three-dimensional lane line recognition device of the above-mentioned embodiment corresponds to the three-dimensional lane line recognition method of the above-mentioned embodiment. Therefore, the part of the three-dimensional lane line recognition device of the above-mentioned embodiment that is not described in detail can be obtained by referring to the content of the three-dimensional lane line recognition method of the above-mentioned embodiment, that is, the specific steps recorded in the three-dimensional lane line recognition method of the above-mentioned embodiment can be understood as the functions that can be realized by the data packaging module and the data acquisition module of the above-mentioned embodiment, and will not be repeated here.
[0067] Furthermore, if the three-dimensional lane recognition device of the above embodiment is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0068] Another embodiment of the present application further proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the three-dimensional lane line recognition method as described in the above embodiment is implemented.
[0069] Specifically, the computer-readable storage medium may include: any entity or recording medium that can carry the computer program instructions, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0070] Another embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the three-dimensional lane line recognition method described in the above embodiment when executing the program.
[0071] Wherein, the electronic device may also include a bus connecting different components (including memory and processor). The memory may include a computer-readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The memory may also include at least one program product, which has a set of (e.g., at least one) program modules, which are configured to perform the functions of the various embodiments of the present application. The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device (e.g., a network card) that enables the electronic device to communicate with one or more other computing devices, such communication may be performed through an input / output (I / O) interface, and the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through a network adapter.
[0072] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A three-dimensional lane line recognition method, characterized in that: include: Acquire a front view image of the vehicle; Inputting the vehicle front view image into a pre-trained three-dimensional lane line recognition model to perform lane line recognition to obtain three-dimensional lane line information; The three-dimensional lane line recognition model includes a backbone network, a spatial transformation network, an encoder, and a decoder connected in sequence. The backbone network is used to receive a front view image of a vehicle, and perform two-dimensional feature encoding on the front view image of the vehicle to obtain a two-dimensional lane line segmentation result; the spatial transformation network is used to convert the two-dimensional lane line segmentation result to a top-view coordinate system; the encoder is used to perform three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system to obtain a three-dimensional lane line feature; The decoder is used to decode the three-dimensional lane line features to obtain three-dimensional lane line information.
2. The method according to claim 1, characterized in that The training of the three-dimensional lane line recognition model includes a first stage and a second stage. The first stage includes obtaining a backbone network to be trained, and pre-training the backbone network to be trained using vehicle front view image samples and their corresponding two-dimensional annotations to obtain a pre-trained backbone network; the second stage includes constructing a three-dimensional lane line recognition model to be trained based on the pre-trained backbone network and the spatial transformation network, encoder, and decoder to be trained, and training the three-dimensional lane line recognition model to be trained using vehicle front view image samples and their corresponding three-dimensional annotations to obtain a three-dimensional lane line recognition model.
3. The method according to claim 2, characterized in that The two-dimensional annotation is a mask image corresponding to the vehicle front view image sample.
4. The method according to claim 2, characterized in that: The three-dimensional annotation is the three-dimensional position information of the lane line in the vehicle front view image sample.
5. The method according to claim 2, characterized in that: The first stage specifically includes: using the two-dimensional annotation as a supervision signal of the backbone network to be trained to train the backbone network to obtain a pre-trained backbone network; The second stage specifically includes: training the encoder and decoder using the three-dimensional annotation as supervisory signals of the encoder and decoder.
6. The method according to claim 5, characterized in that The second stage specifically includes: According to the internal and external parameters of the vehicle's front-view camera, the three-dimensional position information of the lane line in the vehicle's front-view image sample is projected into the two-dimensional space under the viewing angle of the vehicle's front-view camera to generate a two-dimensional annotation, and the pre-trained backbone network is trained using the two-dimensional annotation as a supervision signal of the pre-trained backbone network.
7. A three-dimensional lane line recognition device, characterized in that: include: An image acquisition module, used for acquiring a front view image of a vehicle; An image recognition module is used to input the vehicle front view image into a pre-trained three-dimensional lane line recognition model to perform lane line recognition and obtain three-dimensional lane line information; The three-dimensional lane line recognition model includes a backbone network, a spatial transformation network, an encoder, and a decoder connected in sequence. The backbone network is used to receive a front view image of a vehicle, and perform two-dimensional feature encoding on the front view image of the vehicle to obtain a two-dimensional lane line segmentation result; the spatial transformation network is used to convert the two-dimensional lane line segmentation result to a top-view coordinate system; the encoder is used to perform three-dimensional feature encoding on the lane segmentation result in the top-view coordinate system to obtain a three-dimensional lane line feature; The decoder is used to decode the three-dimensional lane line features to obtain three-dimensional lane line information.
8. The device according to claim 7, characterized in that The training of the three-dimensional lane line recognition model includes a first stage and a second stage. The first stage includes obtaining a backbone network to be trained, and pre-training the backbone network to be trained using vehicle front view image samples and their corresponding two-dimensional annotations to obtain a pre-trained backbone network; the second stage includes constructing a three-dimensional lane line recognition model to be trained based on the pre-trained backbone network and the spatial transformation network, encoder, and decoder to be trained, and training the three-dimensional lane line recognition model to be trained using vehicle front view image samples and their corresponding three-dimensional annotations to obtain a three-dimensional lane line recognition model.
9. The device according to claim 8, characterized in that The two-dimensional annotation is a mask image corresponding to the vehicle front view image sample, and the three-dimensional annotation is the three-dimensional position information of the lane line in the vehicle front view image sample.
10. The device according to claim 8, characterized in that The first stage specifically includes: using the two-dimensional annotation as a supervision signal of the backbone network to be trained to train the backbone network to obtain a pre-trained backbone network; The second stage specifically includes: training the encoder and decoder using the three-dimensional annotation as supervisory signals of the encoder and decoder; According to the internal and external parameters of the vehicle's front-view camera, the three-dimensional position information of the lane line in the vehicle's front-view image sample is projected into the two-dimensional space under the viewing angle of the vehicle's front-view camera to generate a two-dimensional annotation, and the pre-trained backbone network is trained using the two-dimensional annotation as a supervision signal of the pre-trained backbone network.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the three-dimensional lane line recognition method according to any one of claims 1 to 6 is implemented.
12. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the three-dimensional lane line recognition method according to any one of claims 1 to 6 is implemented.