Lane line detection system and method

Abstract

The present invention discloses a lane line detection system and method, the system comprising: a camera signal acquisition module, a laser radar signal acquisition module, a spatiotemporal registration module, a camera signal preprocessing module and a multi-task lane line detection module; the present invention integrates the information of the laser radar and the camera, and can adapt to the lane line detection task under the change of lighting conditions; the lane line detection model is established by using the multi-task concept, and the lane line position is determined and the lane line type is identified at the same time, thereby improving the network efficiency and optimizing the resource allocation.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and specifically relates to a lane line detection system and method. Background Art

[0002] With the increasing number of cars, road traffic has become increasingly dense and complex, which has led to increased driving pressure, reduced drivers' driving ability in normal traffic scenarios, and greatly increased the probability of traffic accidents. Among them, lane changing behavior is one of the important causes of traffic accidents and traffic congestion, especially in urban areas, where the traffic density is high and lane changing collisions are very likely to occur, and even lead to chain rear-end collisions. Compared with human driving, intelligent driving systems have the advantages of short reaction time and high perception accuracy. Therefore, research on intelligent driving technology is of great significance to reducing traffic accidents caused by human factors.

[0003] At present, the research on lane-changing decision technology focuses on the impact of the motion status of other vehicles on the decision. However, under real traffic conditions, the execution of lane-changing decisions must comply with traffic regulations, which requires the system to analyze lane line information. However, in existing technologies, only the existence of lane lines can be classified, while lane-changing decisions require more specific lane line information, such as whether they are dotted lines. LiDAR can work all day and night, and has a longer detection distance than visual sensors, and can provide curvature information; cameras can obtain richer scene information, and in good weather conditions, the detection effect at close range is very accurate. Information fusion of the two can achieve sensor data complementarity, thereby obtaining more complete lane line information and improving the system's lane line detection capabilities. Summary of the invention

[0004] In view of the above-mentioned deficiencies of the prior art, the purpose of the present invention is to provide a lane line detection system and method to solve the problem of lane line detection under harsh lighting conditions in the prior art; the present invention integrates the information of lidar and camera, and can adapt to the lane line detection task under changing lighting conditions; uses the multi-task concept to establish a lane line detection model, and simultaneously determines the lane line position and identifies the lane line type, thereby improving network efficiency and optimizing resource allocation.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A lane line detection system of the present invention comprises: a camera signal acquisition module, a laser radar signal acquisition module, a time-space registration module, a camera signal preprocessing module and a multi-task lane line detection module; wherein,

[0007] The camera signal acquisition module is used to collect image data of the road ahead of the moving vehicle;

[0008] The laser radar signal acquisition module is used to sense the point cloud data of the road scene in front of the moving vehicle;

[0009] The spatiotemporal registration module includes: sensor data time registration and sensor data space registration; the sensor data time registration is used to unify the data acquired by the camera signal acquisition module and the laser radar signal acquisition module into the same time coordinate system; the sensor data space registration is used to convert the data acquired by the camera signal acquisition module and the laser radar signal acquisition module into the same coordinate system;

[0010] The camera signal preprocessing module is used to perform inverse perspective transformation on the road image data after time registration and space registration, and extract the ROI area in the image;

[0011] The multi-task lane line detection module is used to establish a multi-task lane line detection model, which includes: an image feature extraction module, a point cloud feature extraction module, a feature fusion module, a lane line generation module and a lane line classification module;

[0012] The image feature extraction module inputs the preprocessed road image data into the image feature extraction network to obtain the lane line position feature vector F1 and the lane line category feature vector F2;

[0013] The point cloud feature extraction module inputs the road scene point cloud data that has been registered in time and space into the point cloud feature extraction network to obtain the lane line feature vector F3;

[0014] The feature fusion module fuses the lane line position feature vector F1 and the lane line feature vector F3 based on the feature fusion network, dynamically adjusts the attention weights of the laser radar and camera features according to the input feature vectors, and calculates the weighted sum of the attention weights and the original feature vectors as the fused lane line position feature vector;

[0015] The lane line generation module inputs the lane line position feature vector obtained after fusion into the lane line generation network, determines the position of the lane line, and fits the lane line based on the least squares method;

[0016] The lane classification module inputs the fused position feature vector and lane category feature vector F2 into the lane classification network, and outputs the probability distribution of multiple categories of lane detection through the SoftMax function.

[0017] Furthermore, the camera signal preprocessing module is used for inverse perspective transformation and extraction of ROI area; assuming that the height between the camera and the ground is h, the corresponding pitch angle is θ, the yaw angle is α, and f u is the equivalent focal length of the image plane in the u direction, f vis the equivalent focal length of the image plane in the v direction, (c u ,c v ) corresponds to the optical center of the image plane; the above parameters are determined according to the installation position of the camera, and then the corresponding relationship between any point on the road surface and the image coordinate system is determined by the transformation matrix T:

[0018]

[0019] In the formula, a1=cosθ, a2=cosα, b1=sinθ, b2=sinα, and the generalized inverse matrix of the transformation matrix T is solved by singular value decomposition to complete the inverse perspective transformation of the image.

[0020] The lower part of the image after inverse perspective transformation is captured as the ROI area for lane line detection.

[0021] Furthermore, the calculation process of the lane line position feature vector F1 and the lane line category feature vector F2 is: based on the YOLO v3 model, as the image feature extraction network, and adding two MLP layers as the lane line position feature extraction branch and the lane line category feature extraction branch; it is expressed by the following formula:

[0022]

[0023]

[0024] Where F1 is the lane line position feature vector, F2 is the lane line category feature vector, is the image data after preprocessing.

[0025] Furthermore, the calculation process of the lane feature vector F3 is as follows: based on the YOLO v3 model, as a point cloud feature extraction network, the formula is as follows:

[0026]

[0027] Where F3 is the lane feature vector, It is a point cloud set after spatiotemporal registration.

[0028] Furthermore, the lane line categories output by the lane line classification module are: no lane line, solid white line, dashed white line, double solid white line, single solid yellow line, and double solid yellow line.

[0029] The present invention also provides a lane line detection method, the steps are as follows:

[0030] 1) Obtaining an image set I and a point cloud set P in front of the moving vehicle;

[0031] 2) performing temporal registration and spatial registration on the image set I and the point cloud set P;

[0032] 3) Perform inverse perspective transformation on the image after spatiotemporal registration and extract the ROI area in the image;

[0033] 4) Establish a multi-task lane detection model;

[0034] 5) Collecting the image in front of the moving vehicle and the corresponding point cloud data, establishing the real lane line position and category label, which are used to train the multi-task lane line detection model established in step 4), so as to obtain the network parameters of the established lane line detection model, and train the multi-task lane line detection model with the goal of reducing the loss function;

[0035] 6) Use the trained model to detect the current scene, obtain the image and point cloud data in the scene, and after processing in steps 2) and 3), use it as the input of the lane line detection model to obtain the lane line position and lane line category in front of the current moving vehicle.

[0036] Furthermore, the temporal registration of the image set I and the point cloud set P adopts a least squares registration method, an interpolation and extrapolation method or a Lagrange interpolation method.

[0037] Furthermore, the spatial registration of the image set I and the point cloud set P specifically includes: using the image coordinate system, the pixel coordinate system and the world coordinate system as intermediaries to achieve the conversion between the laser radar coordinate system and the camera coordinate system. The point cloud set after the spatiotemporal registration is recorded as Image data is recorded as

[0038] Furthermore, the step 4) specifically includes: inputting the preprocessed road image data into an image feature extraction network to obtain a lane line position feature vector F1 and a lane line category feature vector F2;

[0039] Based on the YOLO v3 model, it is used as the image feature extraction network, and two MLP layers are added as lane position feature extraction branches and lane category feature extraction branches; the formula is as follows:

[0040]

[0041]

[0042] Where F1 is the lane line position feature vector, F2 is the lane line category feature vector, is the image data after preprocessing.

[0043] Furthermore, the step 4) specifically includes: inputting the road scene point cloud data that has been temporally and spatially registered into a point cloud feature extraction network to obtain a lane feature vector F3; based on the YOLO v3 model, as a point cloud feature extraction network, it is expressed by the following formula:

[0044]

[0045] Where F3 is the lane feature vector, It is a point cloud set after spatiotemporal registration.

[0046] Furthermore, the step 4) specifically includes fusing feature vectors based on a feature fusion network: dynamically adjusting the weight value α of the laser radar and camera features i ; The attention weight α of the lidar and camera features i The calculation process is as follows:

[0047] e i =tanh(WF i +UO)

[0048] α i =exp(e i ) / ∑ k exp(e k )

[0049] Where i∈{1,3}, W, U are the coefficient matrices of the attention weights obtained through training, and O is the output matrix of the current lane line position;

[0050] The feature vector after fusion by the feature fusion network is the weighted sum of the original feature vector and the attention weight, which can be expressed by the following formula:

[0051] F=∑ i α i F i

[0052] In the formula, F is the fused feature vector, F i is the original feature vector.

[0053] Furthermore, the step 4) specifically includes determining the lane line position: selecting a fully connected layer to fit the lane line position;

[0054] O=FC(F)

[0055] Where O = {o1, o2, ..., o n}={(x1, y1), (x2, y2),..., (x n ,y n )} is a set of data points to be fitted, and the quadratic curve is selected as the fitted expression:

[0056] y=a0+a1x+a2x 2

[0057] Where a0, a1, a2 are the position parameters to be obtained.

[0058] Furthermore, the step 4) specifically also includes fitting the probability distribution of different lane line categories based on the SoftMax function: P(c)=SoftMax([F, F3]).

[0059] Furthermore, the loss function L in step 5) includes: the center coordinate error function L coor And the classification error function L clas ;in,

[0060] L=L coor +L clas

[0061] In the formula, the center coordinate error function L coor It is used to evaluate the accuracy of the horizontal and vertical coordinates of the center of the target prediction box. The specific calculation is:

[0062]

[0063] In the formula, λ coor is the weight coefficient of coordinate error; B is the number of candidate boxes generated for each grid; is the possibility that a lane line exists in the j-th prediction box of the i-th grid. If so, Otherwise 0; x i ,y i are the horizontal coordinate and total coordinate of the predicted center point of the i-th sliding window, respectively. are the corresponding real horizontal and vertical coordinates respectively; the detected lane lines include 6 categories, namely class = {no lane line, white solid line, white dashed line, white double solid line, single yellow solid line, double yellow solid line}, and the classification error function L clas for:

[0064]

[0065] In the formula, p i and They represent the true probability and predicted probability of the lane line target in the i-th cell respectively.

[0066] Beneficial effects of the present invention:

[0067] The present invention uses a camera and a laser radar sensor to detect the position of lane lines and classify the lane lines;

[0068] On the one hand, it makes up for the limitation of camera detection in night environment. Information fusion of the two can achieve sensor data complementarity, thereby obtaining more complete lane line information and improving the lane line detection capability of the system.

[0069] On the other hand, the lane line results are subdivided to provide more accurate and effective support information for lane change decisions. After obtaining the image in front of the moving vehicle, an inverse perspective transformation is performed and the ROI area is extracted, which can effectively improve the efficiency of lane line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a structural principle diagram of the system of the present invention.

[0071] Figure 2 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0072] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0073] Reference Figure 1 As shown, a lane line detection system of the present invention includes: a camera signal acquisition module, a laser radar signal acquisition module, a spatiotemporal registration module, a camera signal preprocessing module and a multi-task lane line detection module; wherein,

[0074] The camera signal acquisition module is used to collect image data of the road ahead of the moving vehicle;

[0075] The laser radar signal acquisition module is used to sense the point cloud data of the road scene in front of the moving vehicle;

[0076] The spatiotemporal registration module includes: sensor data time registration and sensor data space registration; the sensor data time registration is used to unify the data acquired by the camera signal acquisition module and the laser radar signal acquisition module into the same time coordinate system; the sensor data space registration is used to convert the data acquired by the camera signal acquisition module and the laser radar signal acquisition module into the same coordinate system;

[0077] The camera signal preprocessing module is used to perform inverse perspective transformation on the road image data after time registration and space registration, and extract the ROI area in the image;

[0078] The camera signal preprocessing module is used for inverse perspective transformation and extraction of ROI area; assuming that the height between the camera and the ground is h, the corresponding pitch angle is θ, the yaw angle is α, and f u is the equivalent focal length of the image plane in the u direction, f vis the equivalent focal length of the image plane in the v direction, (c u ,c v ) corresponds to the optical center of the image plane; the above parameters are determined according to the installation position of the camera, and then the corresponding relationship between any point on the road surface and the image coordinate system is determined by the transformation matrix T:

[0079]

[0080] In the formula, a1=cosθ, a2=cosα, b1=sinθ, b2=sinα, and the generalized inverse matrix of the transformation matrix T is solved by singular value decomposition to complete the inverse perspective transformation of the image.

[0081] The multi-task lane line detection module is used to establish a multi-task lane line detection model, which includes: an image feature extraction module, a point cloud feature extraction module, a feature fusion module, a lane line generation module and a lane line classification module;

[0082] The image feature extraction module inputs the preprocessed road image data into the image feature extraction network to obtain the lane line position feature vector F1 and the lane line category feature vector F2;

[0083] The calculation process of the lane line position feature vector F1 and the lane line category feature vector F2 is: based on the YOLO v3 model, as the image feature extraction network, and adding two MLP layers as the lane line position feature extraction branch and the lane line category feature extraction branch; it is expressed as follows:

[0084]

[0085]

[0086] Where F1 is the lane line position feature vector, F2 is the lane line category feature vector, is the image data after preprocessing.

[0087] The point cloud feature extraction module inputs the road scene point cloud data that has been registered in time and space into the point cloud feature extraction network to obtain the lane line feature vector F3;

[0088] The calculation process of the lane feature vector F3 is as follows: Based on the YOLO v3 model, as a point cloud feature extraction network, it is expressed by the following formula:

[0089]

[0090] Where F3 is the lane feature vector, It is a point cloud set after spatiotemporal registration.

[0091] The feature fusion module fuses the lane line position feature vector F1 and the lane line feature vector F3 based on the feature fusion network, dynamically adjusts the attention weights of the laser radar and camera features according to the input feature vectors, and calculates the weighted sum of the attention weights and the original feature vectors as the fused lane line position feature vector;

[0092] The lane line generation module inputs the lane line position feature vector obtained after fusion into the lane line generation network, determines the position of the lane line, and fits the lane line based on the least squares method;

[0093] The lane classification module inputs the fused position feature vector and lane category feature vector F2 into the lane classification network, and outputs the probability distribution of multiple categories of lane detection through the SoftMax function.

[0094] Among them, the lower part of the image after inverse perspective transformation is intercepted as the ROI area for lane line detection.

[0095] Reference Figure 2 As shown, a lane line detection method of the present invention comprises the following steps:

[0096] 1) Obtain an image set I and a point cloud set P in front of a moving vehicle; in the example, the image data and point cloud data in front of the vehicle are collected by a vehicle-mounted camera and a lidar.

[0097] 2) performing temporal registration and spatial registration on the image set I and the point cloud set P; the temporal registration on the image set I and the point cloud set P adopts the least square registration method, the interpolation and extrapolation method or the Lagrange interpolation method;

[0098] The spatial registration of the image set I and the point cloud set P specifically includes: using the image coordinate system, the pixel coordinate system and the world coordinate system as intermediaries to achieve the conversion between the laser radar coordinate system and the camera coordinate system. The point cloud set after the spatiotemporal registration is recorded as Image data is recorded as

[0099] 3) Perform inverse perspective transformation on the image after spatiotemporal registration and extract the ROI area in the image;

[0100] 4) Establish a multi-task lane detection model based on the image feature extraction network, point cloud feature extraction network, feature fusion network, lane generation network and lane classification network; use the data processed in steps 2) and 3) as the input of the network to obtain the position of the lane and the probability distribution of multiple categories of lanes; in the example, it specifically includes:

[0101] The preprocessed road image data is input into the image feature extraction network to obtain the lane line position feature vector F1 and the lane line category feature vector F2;

[0102] Based on the YOLO v3 model, it is used as the image feature extraction network, and two MLP layers are added as lane position feature extraction branches and lane category feature extraction branches; the formula is as follows:

[0103]

[0104]

[0105] Where F1 is the lane line position feature vector, F2 is the lane line category feature vector, is the image data after preprocessing.

[0106] The road scene point cloud data that has been registered in time and space is input into the point cloud feature extraction network to obtain the lane line feature vector F3; based on the YOLO v3 model, as the point cloud feature extraction network, it is expressed by the following formula:

[0107]

[0108] Where F3 is the lane feature vector, It is a point cloud set after space-time registration;

[0109] Fusion of feature vectors based on feature fusion network: Dynamically adjust the weight value α of lidar and camera features i ; The attention weight α of the lidar and camera features i The calculation process is as follows:

[0110] e i =tanh(WF i +UO)

[0111] α i =exp(e i ) / ∑ k exp(e k )

[0112] Where i∈{1,3}, W, U are the coefficient matrices of the attention weights obtained through training, and O is the output matrix of the current lane line position;

[0113] The feature vector after fusion by the feature fusion network is the weighted sum of the original feature vector and the attention weight, which can be expressed by the following formula:

[0114] F=∑ i α i F i

[0115] In the formula, F is the fused feature vector, F i is the original feature vector.

[0116] Determine the lane line position: Select the fully connected layer to fit the lane line position;

[0117] O=FC(F)

[0118] Where O = {o1, o2, ..., o n}={(x1, y1), (x2, y2),..., (x n ,y n )} is a set of data points to be fitted, and the quadratic curve is selected as the fitted expression:

[0119] y=a0+a1x+a2x 2

[0120] Where a0, a1, a2 are the position parameters to be obtained.

[0121] Based on the SoftMax function fitting, the probability distribution of different lane line categories is obtained:

[0122] P(c)=SoftMax([F,F3]).

[0123] 5) Collecting the image in front of the moving vehicle and the corresponding point cloud data, establishing the real lane line position and category label, which are used to train the multi-task lane line detection model established in step 4), so as to obtain the network parameters of the established lane line detection model, and train the multi-task lane line detection model with the goal of reducing the loss function;

[0124] The loss function L in step 5) includes: the center coordinate error function L coor And the classification error function L clas ;in,

[0125] L=L coor +L clas

[0126] In the formula, the center coordinate error function L coor It is used to evaluate the accuracy of the horizontal and vertical coordinates of the center of the target prediction box. The specific calculation is:

[0127]

[0128] In the formula, λ coor is the weight coefficient of coordinate error; B is the number of candidate boxes generated for each grid; is the probability that a lane line exists in the j-th prediction box of the i-th grid. If so, Otherwise 0; xi ,y i are the horizontal coordinate and total coordinate of the predicted center point of the i-th sliding window, respectively. are the corresponding real horizontal and vertical coordinates respectively; the detected lane lines include 6 categories, namely class = {no lane line, white solid line, white dashed line, white double solid line, single yellow solid line, double yellow solid line}, and the classification error function L clas for:

[0129]

[0130] In the formula, p i and They represent the true probability and predicted probability of the lane line target in the i-th cell respectively.

[0131] 6) Use the trained model to detect the current scene, obtain the image and point cloud data in the scene, and after processing in steps 2) and 3), use it as the input of the lane line detection model to obtain the lane line position and lane line category in front of the current moving vehicle.

[0132] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A lane detection system, characterized in that: include: Camera signal acquisition module, lidar signal acquisition module, spatiotemporal registration module, camera signal preprocessing module and multi-task lane line detection module; The camera signal acquisition module is used to collect image data of the road ahead of the moving vehicle; The laser radar signal acquisition module is used to sense the point cloud data of the road scene in front of the moving vehicle; The spatiotemporal registration module includes: sensor data time registration and sensor data space registration; the sensor data time registration is used to unify the data acquired by the camera signal acquisition module and the laser radar signal acquisition module into the same time coordinate system; the sensor data space registration is used to convert the data acquired by the camera signal acquisition module and the laser radar signal acquisition module into the same coordinate system; The camera signal preprocessing module is used to perform inverse perspective transformation on the road image data after time registration and space registration, and extract the ROI area in the image; The multi-task lane line detection module is used to establish a multi-task lane line detection model, which includes: an image feature extraction module, a point cloud feature extraction module, a feature fusion module, a lane line generation module and a lane line classification module; The image feature extraction module inputs the preprocessed road image data into the image feature extraction network to obtain the lane line position feature vector F1 and the lane line category feature vector F2; The point cloud feature extraction module inputs the road scene point cloud data that has been registered in time and space into the point cloud feature extraction network to obtain the lane line feature vector F3; The feature fusion module fuses the lane line position feature vector F1 and the lane line feature vector F3 based on the feature fusion network, dynamically adjusts the attention weights of the laser radar and camera features according to the input feature vectors, and calculates the weighted sum of the attention weights and the original feature vectors as the fused lane line position feature vector; The lane line generation module inputs the lane line position feature vector obtained after fusion into the lane line generation network, determines the position of the lane line, and fits the lane line based on the least squares method; The lane classification module inputs the fused position feature vector and lane category feature vector F2 into the lane classification network, and outputs the probability distribution of multiple categories of lane detection; The calculation process of the lane line position feature vector F1 and the lane line category feature vector F2 is: based on the YOLO v3 model, as the image feature extraction network, and adding two MLP layers as the lane line position feature extraction branch and the lane line category feature extraction branch; it is expressed as follows: Where F1 is the lane line position feature vector, F2 is the lane line category feature vector, is the preprocessed image data; The calculation process of the lane feature vector F3 is as follows: Based on the YOLO v3 model, as a point cloud feature extraction network, it is expressed by the following formula: Where F3 is the lane feature vector, It is a point cloud set after spatiotemporal registration.

2. The lane detection system according to claim 1, characterized in that: The camera signal preprocessing module is used for inverse perspective transformation and extraction of ROI area; assuming that the height between the camera and the ground is h, the corresponding pitch angle is θ, the yaw angle is α, and f u is the equivalent focal length of the image plane in the u direction, f v is the equivalent focal length of the image plane in the v direction, (c u ,c v ) corresponds to the optical center of the image plane; the above parameters are determined according to the installation position of the camera, and then the corresponding relationship between any point on the road surface and the image coordinate system is determined by the transformation matrix T: In the formula, a1=cosθ, a2=cosα, b1=sinθ, b2=sinα, and the generalized inverse matrix of the transformation matrix T is solved by singular value decomposition to complete the inverse perspective transformation of the image.

3. A lane line detection method, characterized in that: Here are the steps: 1) Obtaining an image set I and a point cloud set P in front of the moving vehicle; 2) performing temporal registration and spatial registration on the image set I and the point cloud set P; 3) Perform inverse perspective transformation on the image after spatiotemporal registration and extract the ROI area in the image; 4) Establish a multi-task lane detection model; 5) Collecting the image in front of the moving vehicle and the corresponding point cloud data, establishing the real lane line position and category label, which are used to train the multi-task lane line detection model established in step 4), so as to obtain the network parameters of the established lane line detection model, and train the multi-task lane line detection model with the goal of reducing the loss function; 6) Using the trained model to detect the current scene, obtaining the image and point cloud data in the scene, and after processing in steps 2) and 3), using them as the input of the lane detection model to obtain the lane position and lane category in front of the current vehicle; The step 4) specifically includes: inputting the preprocessed road image data into an image feature extraction network to obtain a lane line position feature vector F1 and a lane line category feature vector F2; Based on the YOLO v3 model, it is used as the image feature extraction network, and two MLP layers are added as lane position feature extraction branches and lane category feature extraction branches; the formula is as follows: Where F1 is the lane line position feature vector, F2 is the lane line category feature vector, is the preprocessed image data; The step 4) specifically includes: inputting the road scene point cloud data that has been registered in time and space into the point cloud feature extraction network to obtain the lane line feature vector F3; based on the YOLO v3 model as the point cloud feature extraction network, it is expressed by the following formula: Where F3 is the lane feature vector, It is a point cloud set after spatiotemporal registration.

4. The lane line detection method according to claim 3, characterized in that: The temporal registration of the image set I and the point cloud set P adopts the least square registration method, the interpolation and extrapolation method or the Lagrange interpolation method.

5. The lane line detection method according to claim 4, characterized in that: The spatial registration of the image set I and the point cloud set P specifically includes: using the image coordinate system, the pixel coordinate system and the world coordinate system as intermediaries to achieve the conversion between the laser radar coordinate system and the camera coordinate system. The point cloud set after the spatiotemporal registration is recorded as Image data is recorded as 6. The lane line detection method according to claim 3, characterized in that: The step 4) specifically includes fusing feature vectors based on a feature fusion network: dynamically adjusting the weight value α of the laser radar and camera features i ; The attention weight α of the lidar and camera features i The calculation process is as follows: e i =tanh(WF i +UO) α i =exp(e i ) / ∑ k exp(e k ) Where i∈{1,3}, W, U are the coefficient matrices of the attention weights obtained through training, and O is the output matrix of the current lane line position; The feature vector after fusion by the feature fusion network is the weighted sum of the original feature vector and the attention weight, which can be expressed by the following formula: F=∑ i a i F i In the formula, F is the fused feature vector, F i is the original feature vector.

Images