Neural Network-Based Lane Detection Method, Device, Equipment and Medium

Abstract

The present invention discloses a lane line detection method, device, equipment and medium based on a neural network. The method includes: acquiring lane line data collected by an autonomous driving vehicle within a preset period to generate a lane line data set; inputting the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set; performing clustering fusion processing on the estimated lane line set according to a target clustering algorithm to obtain each group of calibrated current lane lines and generate a current lane line set; matching the current lane line set with a preset map to obtain a target map set; and detecting whether the current lane line is the actual lane line of the autonomous driving vehicle according to the current lane line set and the target map set. In the embodiment of the present invention, a neural network is constructed to estimate the collected lane line data, and map matching and detection are performed after lane line calibration, greatly improving the accuracy and reliability of lane line detection and reducing detection errors.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more particularly to a lane line detection method, device, equipment and readable storage medium based on a neural network. Background Art

[0002] Currently, a main research point of Advanced Driver Assistance Systems (ADAS) is to improve the safety of the vehicle itself or the vehicle's driving and reduce road accidents. Intelligent vehicles and driverless vehicles are expected to solve road safety, traffic problems and passenger comfort problems. In the research tasks for intelligent vehicles or driverless vehicles, lane line detection is a complex and challenging task. As a main part of the road, lane lines play a role in providing reference for driverless vehicles and guiding safe driving. Lane line detection includes road positioning, the relative position relationship between the vehicle and the road, and the driving direction of the vehicle.

[0003] In the process of conceiving and implementing the present application, the inventors of the present application found that in the current technical solutions, lane line detection is usually achieved based on the images obtained by a camera and the positioning signals provided by a GPS device. However, the accuracy of the lane lines determined by this solution, as well as the position information of the lane lines and the relative position information between the lane lines and the vehicle, is low and cannot meet the driving requirements of autonomous vehicles. That is, there are problems of low accuracy and large errors in the existing technical solutions for lane line detection.

[0004] The foregoing description is for the purpose of providing general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a lane line detection method, device, equipment and readable storage medium based on a neural network, which can solve the problem that the accuracy of lane line detection is low due to large errors in data caused by hardware accuracy, environmental factors or sensor jitter in the existing lane line detection technology.

[0006] To solve the above problems, the first aspect of the embodiments of the present application provides a lane line detection method based on a neural network, which at least includes the following steps:

[0007] Obtain lane line data collected by an autonomous vehicle within a preset period, and generate a lane line data set, where the lane line data set includes at least one group of lane line data;

[0008] Input the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one group of lane lines;

[0009] Cluster and fuse each group of predicted lane lines in the predicted lane line set according to the target clustering algorithm to obtain each group of calibrated current lane lines, and generate a current lane line set;

[0010] Match the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to a group of predicted lane lines;

[0011] Detect whether the current lane line is the actual lane line of an autonomous vehicle according to the current lane line set and the target map set.

[0012] In a possible implementation manner of the first aspect, the lane line detection method based on a neural network further includes:

[0013] Construct three autoencoder convolutional neural networks with different numbers of layers. The three convolutional neural networks form a parallel convolutional neural network. Among them, each convolutional neural network is used to detect different objects, and the objects include background points, solid lane lines or dashed lane lines.

[0014] In a possible implementation manner of the first aspect, the lane line detection method based on a neural network further includes:

[0015] Train the constructed parallel convolutional neural network using a training data set, and adjust the parameters of the convolutional neural network according to the change of the loss function and the convergence situation of the convolutional neural network during the training process;

[0016] Use the verification set pictures to test the detection effect of the trained parallel convolutional neural network, and adjust the parameters of the convolutional neural network according to the actual detection effect of the convolutional neural network.

[0017] In a possible implementation manner of the first aspect, the step of clustering and fusing each group of predicted lane lines in the predicted lane line set according to the target clustering algorithm to obtain each group of calibrated current lane lines and generate a current lane line set includes:

[0018] Obtain multiple groups of predicted lane lines in the predicted lane line set, and perform lane line recognition and detection processing on each group of predicted lane lines respectively to detect the first lane line corresponding to each group of predicted lane lines;

[0019] According to the target clustering algorithm, perform clustering and fusion processing on the first lane lines corresponding to each group of predicted lane lines in sequence to obtain each group of calibrated current lane lines, and generate a current lane line set.

[0020] In a possible implementation manner of the first aspect, after matching the current lane line set with a preset map to obtain a target map set, it further includes:

[0021] Obtain vehicle visual odometer information and detect whether the current lane line is accurate according to the vehicle visual odometer information.

[0022] In a possible implementation manner of the first aspect, the detecting whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set includes:

[0023] Obtain the local map corresponding to the current lane line;

[0024] Extract the position information of the landmark objects in the local map corresponding to the current lane;

[0025] Match and identify the current lane line according to the position information of the landmark objects, and detect whether the current lane line is accurate.

[0026] In a possible implementation manner of the first aspect, after obtaining the lane line data collected by the autonomous vehicle within a preset period and generating a lane line data set, it further includes:

[0027] Extract the lane line semantic information corresponding to the lane line data and the pixel points of the corresponding lane line according to the pre-established semantic segmentation model;

[0028] Convert the pixel points of the lane line to the vehicle body coordinate system;

[0029] Jointly locate the pixel points of the lane line in the vehicle body coordinate system with the data obtained by inertial navigation to obtain the positioning information of each group of lane line data.

[0030] Correspondingly, a second aspect of the embodiments of the present application provides a lane line detection device based on a neural network, including:

[0031] A data acquisition module, configured to obtain lane line data collected by the autonomous vehicle within a preset period and generate a lane line data set, where the lane line data set includes at least one group of lane line data;

[0032] An estimation module, configured to input the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one group of lane lines;

[0033] A calibration module, configured to perform clustering and fusion processing on each group of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each group of calibrated current lane lines and generate a current lane line set;

[0034] A matching module, configured to match the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to a group of estimated lane lines;

[0035] A detection module, configured to detect whether the current lane line is an actual lane line of an autonomous vehicle according to the current lane line set and the target map set.

[0036] In a third aspect of the embodiments of the present application, a computer device is further provided, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the lane line detection method based on a neural network described in any one of the above are implemented.

[0037] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is further provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the lane line detection method based on a neural network described in any one of the above are implemented.

[0038] Implementing the embodiments of the present invention has the following beneficial effects:

[0039] A lane line detection method, device, device, and readable storage medium based on a neural network provided by an embodiment of the present invention, the method includes: obtaining lane line data collected by an autonomous vehicle within a preset period, generating a lane line data set, where the lane line data set includes at least one group of lane line data; inputting the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one group of lane lines; performing clustering fusion processing on each group of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each group of calibrated current lane lines, and generating a current lane line set; matching the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to a group of estimated lane lines; and detecting whether the current lane line is an actual lane line of an autonomous vehicle according to the current lane line set and the target map set. In the embodiment of the present invention, by performing neural network recognition on lane line data and calibrating the recognized estimated lane lines, and by matching with a high-precision map to obtain the local map where the lane line is located and then detecting whether the current lane line is accurate, the problem that large errors in data caused by hardware accuracy, environmental factors, or sensor jitter affect the lane line detection accuracy is solved. While improving the lane line detection efficiency, the accuracy and reliability of lane line detection are greatly improved, the error of lane line detection is effectively reduced, and the situation that the existing detection technology requires high detection conditions and has large detection errors is avoided. Description of the Drawings

[0040] Figure 1Schematic flowchart of a lane line detection method based on a neural network according to an embodiment of the present application;

[0041] Figure 2 Schematic block diagram of a lane line detection device based on a neural network according to an embodiment of the present application;

[0042] Figure 3 Schematic block diagram of a computer device according to an embodiment of the present application.

[0043] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0044] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0045] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0046] The embodiments of the present application can be applied to a server. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0047] First, the application scenarios that the present invention can provide will be introduced. For example, a lane line detection method, device, equipment and readable storage medium based on a neural network are provided, which can realize lane line detection based on a neural network and improve the efficiency and accuracy of lane line detection.

[0048] The first embodiment of the present invention:

[0049] Please refer to Figure 1 .

[0050] As Figure 1 shown, this embodiment provides a lane line detection method based on a neural network, which at least includes the following steps:

[0051] S1. Obtain the lane line data collected by an autonomous vehicle within a preset period, and generate a lane line data set, where the lane line data set includes at least one group of lane line data;

[0052] S2. Input the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one group of lane lines;

[0053] S3. Perform clustering and fusion processing on each group of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each group of calibrated current lane lines, and generate a current lane line set;

[0054] S4. Match the current lane line set with a preset map to obtain a target map set, where the target map set includes target maps corresponding to at least one group of estimated lane lines;

[0055] S5. Detect whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set.

[0056] In the prior art, lane line detection based on a neural network using image and video analysis uses two-dimensional images collected by a camera sensor, which is greatly affected by the environment. Especially under poor imaging conditions, it is easily interfered by non-lane line points and cannot achieve ideal results, far from meeting the technical indicators of L3 and L4 levels of autonomous driving technology.

[0057] In order to solve the above technical problems, this embodiment proposes a lane line detection method based on a neural network. By performing neural network recognition on lane line data, calibrating the recognized estimated lane lines, and matching with a high-precision map to obtain the local map where the lane line is located, and then detecting whether the current lane line is accurate, thereby solving the problem that large errors in data caused by hardware accuracy, environmental factors, or sensor jitter affect the lane line detection accuracy. While improving the lane line detection efficiency, it greatly improves the accuracy and reliability of lane line detection, effectively reduces the error of lane line detection, and avoids the situation where the existing detection technology requires high detection conditions and has a large detection error.

[0058] Specifically, for step S1, it mainly obtains the lane line data collected by an autonomous vehicle within a preset period and generates a lane line data set. The lane line data set includes at least one group of lane line data, and the lane line data includes background points, solid lane lines, or dashed lane lines, etc.

[0059] For step S2, the lane line data set is input into a pre-constructed neural network for lane line recognition, so as to perform predictive recognition on each group of lane line data in the lane line data set, and a predicted lane line set is obtained. The predicted lane line set includes each group of lane line data and the corresponding predicted lane lines.

[0060] For step S3, according to the target clustering algorithm, clustering and fusion processing is performed on each group of predicted lane lines in the predicted lane line set to obtain each group of calibrated current lane lines, thereby generating a current lane line set.

[0061] For step S4, the current lane line set is matched with a preset high-precision map, so as to obtain a target map set where all lane lines in the current lane line set are located, match the position of the current lane line in the high-precision map, and obtain a local map near this position within a preset range at this position. The target map set includes at least one target map corresponding to a group of predicted lane lines.

[0062] For step S5, according to the current lane line set and the target map set, a local map of the map position where the current lane line is located and the current lane line are obtained. According to the position data and features of the current lane line, it is matched with the lane lines in the local map to detect whether the current lane line is accurate, so as to effectively determine whether the current lane line is the actual lane line of the autonomous driving vehicle.

[0063] In a preferred embodiment, the lane line detection method based on a neural network further includes:

[0064] Construct three auto-encoding convolutional neural networks with different numbers of layers. The three convolutional neural networks form a parallel convolutional neural network. Among them, each convolutional neural network is used to detect different objects, and the objects include background points, solid lane lines or dashed lane lines.

[0065] In a specific embodiment, the specific process of step S2 is as follows: construct three auto-encoder convolutional neural networks with different numbers of layers. The first convolutional neural network includes C1 convolutional layers and C1 deconvolutional layers, the second convolutional neural network includes C2 convolutional layers and C2 deconvolutional layers, and the third convolutional neural network includes C3 convolutional layers and C3 deconvolutional layers, where C1 < C2 < C3; use the training data set as the input of each convolutional neural network, and the output is a tensor in the form of [batch_size, Height, Width, 1]. Combine the outputs of the three convolutional neural networks into a tensor predicts in the form of [batch_size, Height, Width, 3] as the output of the entire parallel convolutional neural network. During the combination process, the output of the first convolutional neural network is placed at the front to detect background points, the output of the second convolutional neural network is placed in the middle to detect solid lane lines, and the output of the third convolutional neural network is placed at the end to detect dashed lane lines.

[0066] In a preferred embodiment, the lane line detection method based on a neural network further includes:

[0067] Use the training data set to train the constructed parallel convolutional neural network, and adjust the parameters of the convolutional neural network according to the change of the loss function and the convergence situation of the convolutional neural network during the training process;

[0068] Use the validation set pictures to test the detection effect of the trained parallel convolutional neural network, and adjust the parameters of the convolutional neural network according to the actual detection effect of the convolutional neural network.

[0069] In a specific embodiment, first create a corresponding label tensor labels through one-hot encoding according to the background and different colors marked in the labeled pictures in the training data set. Use the pictures in the training data set and the corresponding label tensor to train the parallel convolutional neural network. Use the mean square error between predicts and labels as the loss function. During the training process, observe the convergence situation of the convolutional neural network through tensorboard, and adjust the learning rate and the value of the batch size M according to the actual convergence situation of the convolutional neural network. Finally, perform morphological processing on the result output by the convolutional neural network. First, perform opening operation to filter out some smaller isolated noise points, and then perform closing operation to fill some small black holes, so as to obtain more accurate and complete lane lines.

[0070] In a preferred embodiment, the fusing the groups of predicted lane lines in the predicted lane line set according to the target clustering algorithm to obtain each group of calibrated current lane lines and generating a current lane line set includes:

[0071] Obtain multiple groups of predicted lane lines in the set of predicted lane lines, and perform lane line recognition and detection processing on each group of predicted lane lines respectively to detect the first lane line corresponding to each group of predicted lane lines;

[0072] According to the target clustering algorithm, perform clustering and fusion processing on the first lane lines corresponding to each group of predicted lane lines successively to obtain the current lane lines after calibration for each group, and generate a set of current lane lines.

[0073] In a specific embodiment, the specific process of calibrating the current lane lines by obtaining multiple groups of lane lines and performing fusion processing includes: obtaining multiple groups of lane lines collected by an image acquisition device, and performing lane line recognition and detection processing on each group of lane lines respectively, so as to extract the initial lane line corresponding to each group of lane lines and generate multiple groups of initial lane lines; according to the target clustering algorithm, perform clustering and fusion processing on the multiple groups of initial lane lines successively to obtain the lane lines after calibration.

[0074] Exemplarily, the target clustering algorithm can be the K-means clustering algorithm or the Gaussian mixture model clustering algorithm. This embodiment does not limit the target clustering algorithm, and those skilled in the art can determine it according to needs.

[0075] Optionally, performing clustering and fusion processing on multiple groups of lane lines to obtain the current lane lines after calibration may specifically include:

[0076] According to the multiple lane line positioning data of the target road, obtain the multiple lane line positioning data at any target position in the target road; according to the target clustering algorithm, perform clustering on the multiple lane line positioning data at any target position to obtain the lane line clustering center at any target position; repeat the steps from obtaining the multiple lane line positioning data at any target position in the target road according to the multiple lane line positioning data of the target road to performing clustering on the multiple lane line positioning data at any target position according to the target clustering algorithm to obtain the lane line clustering center at any target position to obtain the lane line clustering centers at multiple target positions; according to the target algorithm, perform fusion on the lane line clustering centers at the multiple target positions to obtain the lane lines of the target road.

[0077] In addition, when any lane line positioning data deviates from the pre-stored map data by more than a preset threshold, the lane line positioning data will be deleted. Exemplarily, the preset threshold is twice the width of the current road. When the difference between the lane line longitude and latitude in any lane line positioning data and the lane longitude and latitude corresponding to the lane line in the pre-stored map exceeds the preset threshold, it is considered that the lane line positioning data is abnormal data, and the abnormal data will be deleted, thereby further improving the accuracy of lane line positioning and detection.

[0078] In a preferred embodiment, after matching the current lane line set with a preset map to obtain a target map set, the method further includes:

[0079] Obtaining vehicle visual odometer information, and detecting whether the current lane line is accurate according to the vehicle visual odometer information.

[0080] In a specific embodiment, it is also possible to detect whether the current lane line is accurate through visual odometer technology. To detect the current lane line through vehicle visual odometer information, first obtain the visual odometer information of the current vehicle, and then optimize the lane line detection based on neural network and the positioning accuracy of monocular visual odometer based on monocular vision. Visual odometer (VO) uses the image information collected by an on-vehicle camera to restore the 6-degree-of-freedom information of the vehicle body itself, including 3 degrees of freedom of rotation and 3 degrees of freedom of translation. The visual sensor can provide rich perception information, so as to perform accurate lane line detection based on neural network.

[0081] In a preferred embodiment, detecting whether the current lane line is the actual lane line of an autonomous vehicle according to the current lane line set and the target map set includes:

[0082] Obtaining the local map corresponding to the current lane line;

[0083] Extracting the position information of landmark objects in the local map corresponding to the current lane;

[0084] Performing matching recognition on the current lane line according to the position information of landmark objects, and detecting whether the current lane line is accurate.

[0085] In a specific embodiment, step S5 mainly obtains the local map within the preset range of the current lane line; extracts the position information of landmark objects in the local map corresponding to the current lane; performs matching recognition on the current lane line according to the position information of landmark objects, and detects whether the current lane line is accurate.

[0086] Optionally, the specific process of step S5 is as follows: First, obtain local maps corresponding to multiple target road lane lines; exemplarily, the local map represents a map including target road lane lines and other traffic elements, and the other traffic elements may include road surfaces, street lights, plants, the sky, zebra crossings, buildings, etc. The local maps corresponding to multiple target road lane lines can be obtained from a database that pre-stores local maps corresponding to multiple target road lane lines, and the local maps corresponding to multiple target road lane lines can be obtained by fusing image data and sensor data collected by a map acquisition vehicle cruising on the target road lane lines. Secondly, extract the position information of landmark objects in the local maps corresponding to multiple target road lane lines; exemplarily, the landmark objects can be zebra crossings, traffic signs, large billboards. For example, when multiple target roads are at an intersection and there are zebra crossings at the intersections of each target road and the intersection, then the zebra crossing can be selected as the landmark object. When the map acquisition vehicle collects images of the target road, since the map acquisition vehicle is equipped with a positioning system (such as GPS) and an image acquisition device, traffic element image information and longitude and latitude information will be uploaded. Finally, after globally fusing the local maps corresponding to multiple target road lane lines according to the position information of the landmark objects, match and identify the current lane line according to the position information of the landmark objects, and detect whether the current lane line is accurate.

[0087] In a preferred embodiment, after obtaining the lane line data collected by the autonomous vehicle within a preset period and generating a lane line data set, it further includes:

[0088] According to a pre-established semantic segmentation model, extract the lane line semantic information corresponding to the lane line data and the pixel points of the corresponding lane line;

[0089] Convert the pixel points of the lane line to the vehicle body coordinate system;

[0090] Jointly locate the pixel points of the lane line in the vehicle body coordinate system with the data obtained by inertial navigation to obtain the positioning information of each group of lane line data.

[0091] In a specific embodiment, first, according to a pre-established semantic segmentation model, extract the traffic element semantic information of the target road and the pixel points of the corresponding traffic elements, and the traffic elements include lane lines;

[0092] Exemplarily, semantic segmentation in deep learning classifies each pixel in an image, and can accurately extract the pixel points of traffic elements in the image, and can clearly segment lane lines, road surfaces, road edges, street lights, plants, the sky, etc. Secondly, the pixel points of traffic elements are converted to the vehicle body coordinate system. Thirdly, the pixel points of traffic elements in the vehicle body coordinate system are jointly located with the data obtained by inertial navigation to obtain the positioning information of the target road traffic elements. The inertial navigation can be an inertial navigation system (IMU), etc. According to the inertial navigation data, the pose information of the vehicle can be determined. According to the pixel points of traffic elements in the vehicle body coordinate system, more accurate depth information of traffic elements can be obtained, so as to achieve joint positioning.

[0093] A lane line detection method based on a neural network provided in this embodiment includes: obtaining lane line data collected by an autonomous vehicle within a preset period to generate a lane line data set, where the lane line data set includes at least one set of lane line data; inputting the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one set of lane lines; performing clustering and fusion processing on each set of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each set of calibrated current lane lines, and generating a current lane line set; matching the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to the estimated lane lines; and detecting whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set.

[0094] In this embodiment, by performing neural network recognition on lane line data, calibrating the recognized estimated lane lines, and matching with a high-precision map to obtain the local map where the lane lines are located, and then detecting whether the current lane lines are accurate, the problem that large errors in data caused by hardware accuracy, environmental factors, or sensor jitter affect the lane line detection accuracy is solved. While improving the lane line detection efficiency, the accuracy and reliability of lane line detection are greatly improved, the error of lane line detection is effectively reduced, and the situation that the existing detection technology requires high detection conditions and has large detection errors is avoided.

[0095] The second embodiment of the present invention:

[0096] Please refer to Figure 2 。

[0097] As Figure 2 shown, this embodiment provides a lane line detection device based on a neural network, including:

[0098] The data acquisition module 100 is used to obtain the lane line data collected by the autonomous vehicle within a preset period, generate a lane line data set, and the lane line data set includes at least one group of lane line data;

[0099] The estimation module 200 is used to input the lane line data set into a preset neural network for lane line recognition, and obtain an estimated lane line set, and the estimated lane line set includes at least one group of lane lines;

[0100] The calibration module 300 is used to perform clustering fusion processing on each group of estimated lane lines in the estimated lane line set according to the target clustering algorithm, obtain each group of calibrated current lane lines, and generate a current lane line set;

[0101] The matching module 400 is used to match the current lane line set with a preset map to obtain a target map set, and the target map set includes at least one target map corresponding to the estimated lane line;

[0102] The detection module 500 is used to detect whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set.

[0103] In a preferred embodiment, the lane line detection device based on a neural network further includes:

[0104] The neural network construction module is used to construct three autoencoder convolutional neural networks with different numbers of layers. The three convolutional neural networks form a parallel convolutional neural network. Among them, each convolutional neural network is used to detect different objects, and the objects include background points, solid lane lines or dashed lane lines.

[0105] In a preferred embodiment, the lane line detection device based on a neural network further includes:

[0106] The neural network training module is used to train the constructed parallel convolutional neural network using a training data set, and adjust the parameters of the convolutional neural network according to the change of the loss function and the convergence situation of the convolutional neural network during the training process;

[0107] The neural network verification module is used to use verification set pictures to test the detection effect of the trained parallel convolutional neural network, and adjust the parameters of the convolutional neural network according to the actual detection effect of the convolutional neural network.

[0108] In a preferred embodiment, the calibration module 300 may specifically include:

[0109] The recognition unit is used to obtain multiple groups of estimated lane lines in the estimated lane line set, perform lane line recognition and detection processing on each group of estimated lane lines respectively, and detect the first lane line corresponding to each group of estimated lane lines;

[0110] A clustering unit, configured to perform clustering and fusion processing on the first lane lines corresponding to each group of estimated lane lines according to a target clustering algorithm, obtain the current lane lines calibrated for each group, and generate a set of current lane lines.

[0111] In a preferred embodiment, the neural network-based lane line detection device further includes:

[0112] A visual odometer unit, configured to obtain vehicle visual odometer information and detect whether the current lane lines are accurate according to the vehicle visual odometer information.

[0113] In a preferred embodiment, the detection module 500 may specifically include:

[0114] A map acquisition unit, configured to acquire a local map corresponding to the current lane lines;

[0115] A position extraction unit, configured to extract position information of landmark objects in the local map corresponding to the current lane;

[0116] A matching detection unit, configured to perform matching recognition on the current lane lines according to the position information of the landmark objects and detect whether the current lane lines are accurate.

[0117] In a preferred embodiment, the neural network-based lane line detection device may further include a semantic recognition module, and the semantic recognition module includes:

[0118] A semantic segmentation model unit, configured to extract lane line semantic information corresponding to the lane line data and pixel points of the corresponding lane lines according to a pre-established semantic segmentation model;

[0119] A coordinate conversion unit, configured to convert the pixel points of the lane lines to a vehicle body coordinate system;

[0120] A joint positioning unit, configured to perform joint positioning on the pixel points of the lane lines in the vehicle body coordinate system and data obtained by inertial navigation to obtain positioning information of each group of lane line data.

[0121] In this embodiment, first, the lane line data collected by the autonomous driving vehicle within a preset period is obtained through the data acquisition module to generate a lane line data set, where the lane line data set includes at least one group of lane line data; then, the lane line data set is input into a preset neural network through the estimation module for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one group of lane lines; next, the calibration module performs clustering and fusion processing on each group of estimated lane lines in the estimated lane line set according to the target clustering algorithm to obtain each group of calibrated current lane lines and generate a current lane line set; secondly, the matching module matches the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to the estimated lane lines; finally, the detection module detects whether the current lane lines are the actual lane lines of the autonomous driving vehicle according to the current lane line set and the target map set.

[0122] In this embodiment, by performing neural network recognition on the lane line data, calibrating the recognized estimated lane lines, and matching with a high-precision map to obtain the local map where the lane lines are located and then detecting whether the current lane lines are accurate, the problem that large errors in data caused by hardware accuracy, environmental factors, or sensor jitter affect the lane line detection accuracy is solved. While improving the lane line detection efficiency, the accuracy and reliability of lane line detection are greatly improved, the error of lane line detection is effectively reduced, and the situation that the existing detection technology requires high detection conditions and has large detection errors is avoided.

[0123] Referring to Figure 3 , in the embodiment of the present application, a computer device is further provided. The computer device can be a server, and its internal structure can be as Figure 3As shown in the figure. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a lane line detection method based on a neural network. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a lane line detection method based on a neural network. The lane line detection method based on a neural network includes: obtaining lane line data collected by an autonomous driving vehicle within a preset period, generating a lane line data set, where the lane line data set includes at least one set of lane line data; inputting the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one set of lane lines; performing clustering and fusion processing on each set of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each set of calibrated current lane lines, generating a current lane line set; matching the current lane line set with a preset map to obtain a target map set, where the target map set includes target maps corresponding to at least one set of estimated lane lines; and detecting whether the current lane lines are the actual lane lines of the autonomous driving vehicle according to the current lane line set and the target map set.

[0124] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a lane line detection method based on a neural network, including the steps of: obtaining lane line data collected by an autonomous driving vehicle within a preset period, generating a lane line data set, where the lane line data set includes at least one set of lane line data; inputting the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one set of lane lines; performing clustering and fusion processing on each set of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each set of calibrated current lane lines, generating a current lane line set; matching the current lane line set with a preset map to obtain a target map set, where the target map set includes target maps corresponding to at least one set of estimated lane lines; and detecting whether the current lane lines are the actual lane lines of the autonomous driving vehicle according to the current lane line set and the target map set.

[0125] The above-mentioned lane line detection method based on a neural network identifies lane line data through a neural network, calibrates the estimated lane lines obtained from the identification, matches them with a high-precision map, and detects whether the current lane lines are accurate after obtaining the local map where the lane lines are located, thereby solving the problem that large errors in data caused by hardware precision, environmental factors, or sensor jitter affect the accuracy of lane line detection. While improving the efficiency of lane line detection, it greatly improves the accuracy and reliability of lane line detection, effectively reduces the error of lane line detection, and avoids the situation where the existing detection technology requires high detection conditions and has a large detection error.

[0126] In the above embodiments of the present invention, the descriptions of the various embodiments each have their own emphasis. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0127] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the modules can be a logical function division. In actual implementation, there can be other division methods. For example, multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0128] The modules described as separate components may or may not be physically separated. The components displayed as modules may or may not be physical modules, that is, they can be located in one place or distributed to multiple modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, in each embodiment of the present invention, the functional modules can be integrated in a processing module, or each module can exist physically alone, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0130] The above is the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and deformations can be made, and these improvements and deformations are also regarded as the protection scope of the present invention.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, storage, database, or other medium provided in this application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Claims

1. A lane line detection method based on a neural network, characterized in that, At least include the following steps: Obtain the lane line data collected by the autonomous vehicle within a preset period, and generate a lane line data set, where the lane line data set includes at least one set of lane line data; Input the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one set of lane lines; Perform clustering and fusion processing on each set of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each set of calibrated current lane lines, and generate a current lane line set; Match the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to the estimated lane line; Detect whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set; After matching the current lane line set with the preset map to obtain the target map set, it further includes: Obtain vehicle visual odometer information, and detect whether the current lane line is accurate according to the vehicle visual odometer information.

2. The lane line detection method based on a neural network according to claim 1, wherein It further includes: Construct three autoencoder convolutional neural networks with different numbers of layers. The three convolutional neural networks form a parallel convolutional neural network. Among them, each convolutional neural network is used to detect different objects, and the objects include background points, solid lane lines or dashed lane lines.

3. The lane line detection method based on a neural network according to claim 1, wherein It further includes: Use a training data set to train the constructed parallel convolutional neural network, and adjust the parameters of the convolutional neural network according to the change of the loss function and the convergence situation of the convolutional neural network during the training process; Use validation set pictures to test the detection effect of the trained parallel convolutional neural network, and adjust the parameters of the convolutional neural network according to the actual detection effect of the convolutional neural network.

4. The lane line detection method based on a neural network according to claim 1, characterized in that, The performing clustering and fusion processing on each set of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each set of calibrated current lane lines, and generate a current lane line set includes: Obtain multiple sets of estimated lane lines in the estimated lane line set, and perform lane line recognition and detection processing on each set of estimated lane lines respectively to detect the first lane line corresponding to each set of estimated lane lines; According to the target clustering algorithm, perform clustering and fusion processing on the first lane lines corresponding to each set of estimated lane lines in sequence to obtain each set of calibrated current lane lines, and generate a current lane line set.

5. The lane line detection method based on a neural network according to claim 1, characterized in that The detecting whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set includes: Obtain the local map corresponding to the current lane line; Extract the position information of landmark objects in the local map corresponding to the current lane; Perform matching recognition on the current lane line according to the position information of the landmark objects to detect whether the current lane line is accurate.

6. The lane line detection method based on a neural network according to claim 1, characterized in that After obtaining the lane line data collected by the autonomous vehicle within a preset period and generating a lane line data set, it further includes: Extract the lane line semantic information corresponding to the lane line data and the pixel points of the corresponding lane line according to a pre-established semantic segmentation model; Convert the pixel points of the lane line to the vehicle body coordinate system; Jointly locate the pixel points of the lane lines in the vehicle body coordinate system and the data obtained by inertial navigation to obtain the positioning information of each group of lane line data.

7. A lane line detection device based on a neural network, characterized in that, Including: A data acquisition module, configured to obtain lane line data collected by an autonomous vehicle within a preset period, and generate a lane line data set, where the lane line data set includes at least one group of lane line data; An estimation module, configured to input the lane line data set into a preset neural network for lane line recognition to obtain an estimated lane line set, where the estimated lane line set includes at least one group of lane lines; A calibration module, configured to perform clustering and fusion processing on each group of estimated lane lines in the estimated lane line set according to a target clustering algorithm to obtain each group of calibrated current lane lines, and generate a current lane line set; A matching module, configured to match the current lane line set with a preset map to obtain a target map set, where the target map set includes at least one target map corresponding to an estimated lane line. After obtaining the target map set, it further includes: Obtain vehicle visual odometer information, and detect whether the current lane line is accurate according to the vehicle visual odometer information; A detection module, configured to detect whether the current lane line is the actual lane line of the autonomous vehicle according to the current lane line set and the target map set.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the neural network-based lane line detection method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the neural network-based lane line detection method according to any one of claims 1 to 6 are implemented.

Images