A method and related device for detecting lane lines

Abstract

This application provides a lane line detection method and related apparatus to effectively suppress noise in lane line feature points, thereby improving the accuracy of lane line detection. Specifically, the method includes: the lane line detection apparatus acquiring initial lane line feature points and prior road structure information during vehicle travel; then, when the lane line detection apparatus determines, according to a road prior decision rule, that the prior road structure information is valid and the correlation between the initial lane line feature points and the prior road structure information reaches a preset threshold, converting the initial lane line feature points into first lane line feature points in a road edge coordinate system; then, inputting the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, whereby the Gaussian mixture model is used to classify and denoise the first lane line feature points; finally, outputting the final lane line based on the second lane line feature points.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method and apparatus for detecting lane lines. Background Technology

[0002] Intelligent driving technologies (such as autonomous driving and assisted driving) rely on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and global positioning systems to enable vehicles to achieve autonomous driving without active human intervention. The first step in autonomous driving is the collection and processing of environmental information; therefore, the environmental perception module serves as the interaction port between the intelligent platform and the traffic scenario, and is also a key input to the intelligent platform's motion decision-making and planning control modules. Figure 1 In the autonomous driving system shown, the environmental perception module uses sensor data to acquire traffic scene information. It acts as the "eyes" and "ears" of the autonomous driving system, directly determining the safety and stability of the autonomous vehicle during driving tasks. Within the environmental perception module, lane line detection is one of its core tasks. Lane line detection provides local lane perception information for vehicle motion planning and plays a crucial role in the lane keeping (cruise) system and lane departure warning system of the intelligent vehicle platform.

[0003] Current lane detection technologies mainly offer the following approaches: One approach involves segmenting the target region containing the ground from LiDAR point cloud data, extracting ground point cloud data from the target region, filtering lane line feature points from the ground point cloud data, fitting multiple lane lines based on these feature points, and selecting lane lines whose distances are within a preset range as the target lane lines. Another approach establishes a 3D coordinate system with the LiDAR center as the origin, sets the region of interest, detects the road edge, extracts candidate points based on the geometric features of the road edge, and removes noise based on the consistent direction of the road edge. The spatial location of the road edge determines the point cloud data of the road surface where the lane line is located. Based on the different reflection intensities of different media, the scan lines are layered, and candidate points are extracted by setting a reflection intensity threshold. Density clustering and noise reduction are performed based on the global continuity of the lane lines, and finally, a binomial curve is used to fit the lane line output.

[0004] However, both of these methods have poor noise resistance and poor lane line detection accuracy when dealing with dashed lines, irregular lane lines, uneven lane surfaces, or lane lines with obstacles. Summary of the Invention

[0005] This application provides a lane line detection method and related apparatus to effectively suppress noise at lane line feature points, thereby improving the accuracy of lane line detection.

[0006] In a first aspect, this application provides a lane line detection method, specifically comprising: the lane line detection device acquiring initial lane line feature points and prior road structure information during vehicle travel; then, when the lane line detection device determines, according to a road prior decision rule, that the prior road structure information is valid and the correlation between the initial lane line feature points and the prior road structure information reaches a preset threshold, converting the initial lane line feature points into first lane line feature points in a road edge coordinate system; then, inputting the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, wherein the Gaussian mixture model is used to classify and denoise the first lane line feature points; and finally, outputting the final lane line based on the second lane line feature points.

[0007] In this embodiment, the lane line detection device can acquire in real time the perception results of the road structure (i.e., the prior information of the road structure) and the initial lane line feature points during vehicle driving based on technologies such as laser sensing, radar sensing, or spatial positioning.

[0008] In the technical solution provided in this embodiment, the lane line detection device performs road prior judgment on the road structure prior information and the initial lane line feature points, thereby pre-screening the initial lane line feature points and the road structure prior information. Then, it establishes a road edge coordinate system using the road structure prior information and transforms the initial lane line feature points within this coordinate system, thus ensuring the validity of the initial lane line feature points and the road structure prior information. Finally, the data is input into a Gaussian mixture model for classification and denoising. Since the Gaussian mixture model can effectively learn the spatial distribution characteristics of lane lines of corresponding types of roads, it effectively classifies and denoises the lane line feature points, ultimately improving the accuracy of lane line detection.

[0009] Optionally, the road prior decision rules include, but are not limited to:

[0010] The length of the road structure exceeds a first preset value. This means that the length of the road structure needs to be sufficiently comprehensive to reflect the road structure and its direction of extension.

[0011] The endpoints of the curve of the road structure meet preset conditions. The preset conditions are that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is less than a second preset value, and the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is greater than a third preset value, wherein the second preset value is less than the third preset value.

[0012] The parallelism of the road edges on both sides of the road structure is greater than the parallelism threshold. That is, in a real-world scenario, two road edges on the left and right sides of a road should generally be detected. Considering that the road edge curves are not all parallel, the parallelism of the left and right road edges should be calculated.

[0013] The coupling degree between the road structure and the feature points of the edge lane lines is greater than the coupling degree threshold. The distribution of the road edge and the lane lines in the road structure are mostly consistent, but there are a few cases of inconsistency. Therefore, the coupling degree between the feature points of the outermost lane lines and the road edge curve should be taken into account.

[0014] In one possible implementation, the lane line detection device inputs the first lane line feature points into the Gaussian mixture model to obtain a confidence score. This confidence score indicates the probability value of the second lane line feature point being a valid feature point. That is, when the Gaussian mixture model outputs a valid feature point, it also outputs the probability value of that valid feature point. For example, feature points with a probability value greater than 0.6 in the first lane line feature points are used as the second lane line feature points. Therefore, the second lane line feature points also include feature points with a probability value of 0.7 and feature points with a probability value of 0.8.

[0015] In one possible implementation, if the lane line detection device uses laser sensing technology, the following technical solution can be adopted when acquiring the initial lane line feature points and prior information about the road structure:

[0016] The lane line detection device can acquire prior information about the road structure and the original laser point cloud data of the road surface based on the laser sensing; then, it preprocesses the original laser point cloud data to extract the initial lane line feature points.

[0017] In one possible implementation, the lane line detection device uses a Bayesian classifier to obtain effective lane line feature points from the second lane line feature points and prior clues. The prior clues include, but are not limited to, the confidence level, the similarity between the second lane line feature points and historical tracking results, and the probability value of the absolute value of the reflection intensity of the second lane line feature points. The absolute value of the reflection intensity is used to indicate the relative change characteristics of the point cloud reflectivity intensity. Finally, the effective lane line feature points are optimized using a cost function and a nonlinear optimization method to output the final lane line.

[0018] Based on the above scheme, in one possible implementation, the cost function is designed according to preset rules; the preset rules include at least: 1. The spatial distribution of lane lines and the spatial distribution of road structure are consistent. In this embodiment, the consistency between the spatial distribution of lane lines and the spatial distribution of road structure can be reflected in the parallel similarity between the road curb and the lane line curve. 2. The lane line output of continuous frame data has continuity and consistency, that is, the fit between the lane line feature points and the parametric equation (used to represent the final lane line). In this embodiment, the fit between the lane line feature points and the parametric equation can be reflected in the average point-to-line distance from the lane line feature points to the parametric equation.

[0019] In this embodiment, the lane line detection device can also train the Gaussian mixture model offline. Specifically, this includes: the lane line detection device acquiring an offline dataset; and then training the Gaussian mixture model based on the offline dataset.

[0020] It is understandable that other devices can be used to train the Gaussian mixture model, and then the trained Gaussian mixture model can be transferred to the lane detection device. The specific method is not limited here.

[0021] Based on the above scheme, in one possible implementation, the lane line detection device acquires the offline dataset by including:

[0022] The lane line detection device acquires offline laser point cloud data and offline road structure prior information; then, the lane line detection device preprocesses the offline laser point cloud data to obtain offline lane line feature points; when the correlation between the offline lane line feature points and the offline road structure prior information reaches a preset threshold according to the road prior decision rule, the lane line detection device converts the offline lane line feature points and the offline road structure prior information into the offline dataset in the road edge coordinate system.

[0023] In one possible implementation, the lane line detection device trains the Gaussian mixture model based on the offline dataset by: constructing the Gaussian mixture model based on the spatial distribution of the offline lane line feature points and the prior information of the offline road structure, and solving for the model parameter values ​​of the Gaussian mixture model.

[0024] In one possible implementation, during the online lane detection process, the lane detection device can also update the model parameter values ​​in the Gaussian mixture model based on the output of valid lane feature points from the second lane feature point set. That is, in the online portion, the lane detection device can also update the model parameter values ​​(including the mean and variance of each sub-model of the Gaussian mixture model and the weights of the sub-models) based on the distribution of the valid lane feature points finally output by the Gaussian mixture model in the road edge coordinate system. This allows the Gaussian mixture model to learn the spatial distribution characteristics of lane feature points, thereby improving the accuracy of the Gaussian mixture model in classification and denoising.

[0025] Secondly, this application provides a training method for a Gaussian mixture model, specifically including: acquiring offline laser point cloud data and offline road structure prior information; preprocessing the offline laser point cloud data to obtain offline lane line feature points; when the offline road structure prior information is determined to be valid according to the road prior decision rule and the correlation between the offline lane line feature points and the offline road structure prior information reaches a preset threshold, the lane line detection device converts the offline lane line feature points and the offline road structure prior information into the offline dataset in the road edge coordinate system; constructing the Gaussian mixture model based on the spatial distribution of the offline lane line feature points and the offline road structure prior information, and solving for the model parameter values ​​of the Gaussian mixture model.

[0026] Thirdly, this application provides a lane line detection device that performs the functions described in the first aspect of the application. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned function.

[0027] In one possible implementation, the device includes units or modules for performing the steps of the first aspect above. For example, the device includes: an acquisition module for acquiring prior information about the road structure and initial lane line feature points;

[0028] The processing module is used to, when determining that the prior information of the road structure is valid according to the prior decision rules and the correlation between the initial lane line feature points and the prior information of the road structure reaches a preset threshold, convert the initial lane line feature points into first lane line feature points in the road edge coordinate system; input the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, wherein the Gaussian mixture model is used to classify and denoise the first lane line feature points; and output the final lane line based on the second lane line feature points.

[0029] Optionally, a storage module may also be included to store the necessary program instructions and data for the lane line detection device.

[0030] In one possible implementation, the device includes a processor and a transceiver, the processor being configured to support the lane line detection device in performing the corresponding functions of the method provided in the first aspect above. The transceiver is used to instruct communication between the lane line detection device and other devices. Optionally, the device may also include a memory coupled to the processor, which stores necessary program instructions and data for the lane line detection device.

[0031] In one possible implementation, when the device is a chip within a lane line detection device, the chip includes a processing module and a transceiver module. The transceiver module may be, for example, an input / output interface, pins, or circuitry on the chip, transmitting various types of information acquired by the sensor to other chips or modules coupled to this chip. The processing module may be, for example, a processor, which, when determining that the road structure prior information is valid according to road prior decision rules and that the correlation between the initial lane line feature points and the road structure prior information reaches a preset threshold, transforms the initial lane line feature points into first lane line feature points in the road edge coordinate system; inputs the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, the Gaussian mixture model being used to classify and denoise the first lane line feature points; and outputs the final lane line based on the second lane line feature points. This processing module can execute computer execution instructions stored in a storage unit to support the lane line detection device in performing the method provided in the first aspect described above. Optionally, the storage unit can be a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, such as random access memory (RAM).

[0032] In one possible implementation, the device includes a communication interface and a logic circuit. The communication interface is used to acquire prior information about the road structure and initial lane line feature points. The logic circuit is used to convert the initial lane line feature points into first lane line feature points in the road edge coordinate system when the prior information about the road structure is determined to be valid according to a road prior decision rule and the correlation between the initial lane line feature points and the prior information about the road structure reaches a preset threshold. The first lane line feature points are then input into a Gaussian mixture model to obtain second lane line feature points, whereby the Gaussian mixture model is used to classify and denoise the first lane line feature points. Finally, the device outputs the final lane lines based on the second lane line feature points.

[0033] The processor mentioned above can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs for the data transmission methods described above.

[0034] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions for performing any possible implementation of the method described in any of the above aspects.

[0035] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in any of the above aspects.

[0036] Sixthly, this application provides a chip system including a processor for supporting a lane detection device in performing the functions involved in the foregoing aspects, such as generating or processing the data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing necessary program instructions and data for the lane detection device to perform the functions of any of the foregoing aspects. The chip system may be composed of chips or may include chips and other discrete devices.

[0037] Seventhly, embodiments of this application provide an intelligent vehicle, the system including the lane line detection device described above. Attached Figure Description

[0038] Figure 1 An exemplary framework diagram of an autonomous driving system;

[0039] Figure 2 This is a flowchart illustrating an existing lane line detection method;

[0040] Figure 3 This is a flowchart illustrating another existing lane line detection method;

[0041] Figure 4a A schematic diagram of a lidar sensing framework;

[0042] Figure 4b This is a schematic diagram of a lane line detection device in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of one embodiment of the lane line detection method in this application;

[0044] Figure 6 This is a schematic diagram of the laser-driven roadside segment and the feature points of the side lane line in an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of the road edge coordinate system in an embodiment of this application;

[0046] Figure 8This is a schematic diagram showing the transformation of the initial lane line feature points into the first lane line feature points in the road edge coordinate system in an embodiment of this application.

[0047] Figure 9 This is a schematic diagram illustrating the processing flow of the Gaussian mixture model in offline and online states in the embodiments of this application;

[0048] Figure 10 This is a schematic diagram of the transformation of offline datasets in the road edge coordinate system in an embodiment of this application;

[0049] Figure 11 This is a schematic diagram illustrating the construction of a Gaussian mixture model in an embodiment of this application;

[0050] Figure 12 This is a point cloud feature map output under the Gaussian mixture model in the embodiments of this application;

[0051] Figure 13 This is a schematic diagram of a processing flow for obtaining the final lane line using a Bayesian classifier and nonlinear optimization in an embodiment of this application.

[0052] Figure 14 This is a schematic diagram of a lane line detection device in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0054] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved. The division of units in this application is a logical division. In practical applications, there may be other division methods. For example, multiple units may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the shown or discussed mutual coupling, direct coupling, or communication connection may be through some interface, and the indirect coupling or communication connection between units may be electrical or other similar forms, none of which are limited in this application. Furthermore, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed among multiple circuit units. Some or all of the units can be selected to achieve the purpose of the solution in this application according to actual needs.

[0055] Intelligent driving technologies (such as autonomous driving and assisted driving) rely on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and global positioning systems to enable vehicles to achieve autonomous driving without active human intervention. The first step in autonomous driving is the collection and processing of environmental information; therefore, the environmental perception module serves as the interaction port between the intelligent platform and the traffic scenario, and is also a key input to the intelligent platform's motion decision-making and planning control modules. Figure 1 In the autonomous driving system shown, the environmental perception module uses sensor data to acquire traffic scene information. It acts as the "eyes" and "ears" of the autonomous driving system, directly determining the safety and stability of the autonomous vehicle during driving tasks. Within the environmental perception module, lane line detection is one of its core tasks. Lane line detection provides local lane perception information for vehicle motion planning and plays a crucial role in the lane keeping (cruise) system and lane departure warning system of the intelligent vehicle platform.

[0056] Current lane detection technologies mainly offer the following types: Figure 2 The flowchart shown illustrates one approach: segmenting the target region containing the ground from LiDAR point cloud data, extracting ground point cloud data from the target region, filtering lane line feature points from the ground point cloud data, fitting multiple lane lines based on these feature points, and selecting lane lines whose distances are within a preset range as target lane lines. In this approach, after extracting the ground region, lane line feature point detection, classification, and fitting are directly performed within the ground region. The fitted lane lines are simply filtered out using the preset lane line widths, and the final detection result is output. Therefore, this approach has poor noise resistance and poor detection performance for dashed or irregular lane lines.

[0057] like Figure 3 The flowchart shown illustrates another approach: A three-dimensional coordinate system is established with the LiDAR center as the origin. A region of interest is defined, and the road edge is detected. Candidate points for the road edge are extracted based on its geometric features, and noise is removed based on the consistent direction of the road edge. The spatial location of the road edge determines the point cloud data of the lane lines. Based on the different reflection intensities of different media, the scan lines are layered, and a reflection intensity threshold is set to extract candidate points for the lane lines. Density clustering is performed based on the global continuity of the lane lines, and noise is removed. Finally, a binomial curve is used to fit the lane lines for output. In this approach, the road edge information is used as prior information, merely as a reference for clustering lane line feature points. The point cloud clustering results obtained from the density-based clustering algorithm (DBSCAN) are then aggregated again. However, due to the sparsity of the lane line LiDAR feature points, this method has a short detection range and poor noise resistance. When the road surface is uneven or there are obstacles interfering, the lane line reflection points contain a large amount of noise, resulting in poor accuracy of the lane lines fitted after clustering.

[0058] To address this issue, this application provides a lane line detection method, specifically comprising: the lane line detection device acquiring initial lane line feature points and prior road structure information during vehicle travel; then, when the lane line detection device determines, according to a road prior decision rule, that the prior road structure information is valid and the correlation between the initial lane line feature points and the prior road structure information reaches a preset threshold, converting the initial lane line feature points into first lane line feature points in a road edge coordinate system; then, inputting the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, wherein the Gaussian mixture model is used to classify and denoise the first lane line feature points; finally, outputting the final lane line based on the second lane line feature points.

[0059] This application involves many related knowledge about lane line detection. In order to better understand the solution of this application, the relevant concepts and background knowledge that may be involved in this application will be introduced below.

[0060] (1) Wheeled mobile equipment

[0061] This is a comprehensive system integrating environmental perception, dynamic decision-making and planning, behavior control and execution, and can also be called a wheeled mobile robot or wheeled intelligent agent. For example, it can be wheeled construction equipment, autonomous vehicles, assisted driving vehicles, etc. Any device with wheeled mobility is called a wheeled mobile device as described in this application. For ease of understanding, in the following embodiments of this application, the wheeled mobile device is described as an autonomous vehicle. Autonomous vehicles can be cars, trucks, motorcycles, buses, ships, airplanes, helicopters, lawnmowers, recreational vehicles, amusement park vehicles, construction equipment, trams, golf carts, trains, and handcarts, etc. The embodiments of this application do not impose any special limitations.

[0062] (2) Perception

[0063] Ensuring the accuracy of perception is the primary condition for the safe operation of autonomous driving. From the perspective of sensors, perception can take many forms, such as laser perception modules, visual perception modules, and millimeter-wave perception modules. As one of the key modules, the laser perception module is widely used in systems such as Advanced Driver Assistant Systems (ADAS) and Autonomous Driving Systems (ADS). It can provide wheeled mobile devices equipped with this system (such as autonomous vehicles) with relevant information about the surrounding environment (such as lane markings on the road). This relevant information can also be called perception information, which provides a solid basis for making reasonable regulatory decisions.

[0064] In one possible implementation, embodiments of this application detect lane line information based on LiDAR data. This can be applied to applications such as... Figure 4a The lane detection module of the lidar perception framework shown in the diagram utilizes laser sensing information. A laser beam is emitted in a fixed direction by a laser sensing module (e.g., a laser sensor or lidar mounted on a wheeled mobile device). When the emitted laser encounters an obstacle, it is reflected, thus providing the time difference between emission and reception. Multiplying this time difference by the speed and dividing by two yields the distance from the sensor to the nearest obstacle in that direction, resulting in laser point cloud data. Based on this laser point cloud data, the lidar perception framework uses a laser curb detection module, a laser lane detection module, an object detection module, and a drivable area detection module to obtain laser perception results, which are then sent to other modules.

[0065] The technical solutions provided in the embodiments of this application can be applied to, for example... Figure 4b The lane line detection device shown mainly consists of two parts: sensors and a computing platform. The sensors include a LiDAR sensor and a navigation system. The LiDAR sensor is primarily used to acquire 3D point cloud data of the environment, characterizing the surrounding environment of the intelligent vehicle platform; the computing platform is mainly used to run sensor drivers and various perception algorithms; and the navigation system is mainly used to acquire the pose information of the intelligent vehicle.

[0066] In the computing platform, the lidar driver acquires the raw radar signal and parses it into 3D point cloud location and reflection intensity information, which is then sent to the lidar curb sensing module and the lidar lane detection module. The lidar curb sensing module primarily outputs road structure information as important prior information for the lidar lane detection module. The lidar lane tracking module receives single-frame detection information and vehicle pose information, performs multi-frame tracking processing, and finally outputs a smooth and stable lane perception result to the lane receiving module. The lane receiving module can be a fusion module, motion planning module, prediction module, etc.

[0067] Please refer to details. Figure 5 As shown, one embodiment of the lane line detection method in this application includes:

[0068] 501. The detection device for the lane line acquires laser point cloud data.

[0069] The lane detection device uses lidar to acquire laser point cloud data during the real-time driving process of the vehicle.

[0070] 502. The lane line detection device obtains initial lane line feature points and prior information about the road structure based on the laser point cloud data.

[0071] The lane line detection device denoises and transforms the laser point cloud data, then obtains preprocessed laser point cloud data and point cloud feature maps through parallel projection and spherical projection. Next, it performs ground segmentation on the preprocessed laser point cloud data and the point cloud feature map to obtain road surface point cloud data from the laser point cloud data. Finally, it extracts the initial lane line feature points from the road surface point cloud data using the reflection intensity variation characteristics of the laser point cloud. Simultaneously, it performs image segmentation based on the preprocessed laser point cloud data and the point cloud feature map, and obtains prior information about the road structure using a laser road edge sensing algorithm.

[0072] 503. When the lane detection device determines, according to the road prior decision rules, that the correlation between the initial lane feature point and the prior information of the road structure reaches a preset threshold, the initial lane feature point is transformed into the first lane feature point in the road edge coordinate system.

[0073] In order to detect whether the prior road structure information output by the laser curb sensing algorithm is valid, the lane line detection device presets a prior road decision rule, which includes, but is not limited to:

[0074] 1. The length of the road structure exceeds the first preset value. That is, it is necessary to ensure that the length of the road structure is sufficient to fully reflect the road structure and the information on the direction of road extension.

[0075] 2. The endpoint positions of the curves of the road structure meet the preset conditions. The preset conditions are that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is less than a second preset value, and the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is greater than a third preset value, wherein the second preset value is less than the third preset value.

[0076] 3. The parallelism of the road edges on both sides of the road structure is greater than the parallelism threshold. That is, in a real-world scenario, two road edges on the left and right sides of a road should generally be detected. Considering that the road edge curves are not all parallel, the parallelism of the left and right road edges should be calculated.

[0077] 4. The coupling degree between the road structure and the feature points of the edge lane lines is greater than the coupling degree threshold. The distribution of the road edge and the lane lines in the road structure are mostly consistent, but there are a few cases of inconsistency. Therefore, the coupling degree between the feature points of the outermost lane lines and the road edge curve should be considered.

[0078] Under this road decision rule, the correlation between the initial lane line feature points and the prior information of the road structure is determined. Only when at least the above four constraints are met, and the correlation between the initial lane line feature points and the prior information of the road structure reaches a preset threshold, can the prior information of the road structure be determined to be valid. The laser-guided roadside segment and the side lane line feature points can then be determined as follows: Figure 6 As shown. At this time, the lane detection device transforms the initial lane feature point into the first lane feature point in the road edge coordinate system based on the prior information of the road structure.

[0079] In this embodiment, as Figure 7As shown, the road edge coordinate system is defined as follows: the origin is the perpendicular point from the current vehicle coordinate system origin to the nearest side edge of the road, the s-coordinate is the curve length along the road edge curve, and the d-coordinate is the radial distance from the point to the road edge curve. Therefore, the process of transforming the initial lane line feature point into the first lane line feature point in the road edge coordinate system can be as follows: Figure 8 As shown. Figure 8 The left side shows the initial lane line feature points obtained based on laser sensing results. Figure 8 The right side represents the feature point of the first lane line in the road edge coordinate system.

[0080] 504. The lane line detection device inputs the first lane line feature point into a Gaussian mixture model to obtain the second lane line feature point.

[0081] The lane line detection device inputs the first lane line feature points into a Gaussian mixture model for classification and denoising to obtain the second lane line feature points.

[0082] In this embodiment, the Gaussian mixture model can be expressed in the form shown in the following equation:

[0083]

[0084]

[0085] 0≤π k ≤1

[0086] Where: x is the radial coordinate of the first lane feature point in the road edge coordinate system, π is the model mixing coefficient, μ and ∑ are the mean and variance of the Gaussian distribution, respectively, and K is the number of Gaussian distributions in the Gaussian mixture model.

[0087] In calculating the validity of the first lane feature point in the road edge coordinate system using a Gaussian mixture model, the probability of it belonging to each lane can be output. The class with the highest probability is selected, and the variance relative to the Gaussian distribution in this class is calculated. A variance threshold is set; if the calculated variance is less than or equal to the threshold, the feature point is considered valid in the road prior; otherwise, it is considered invalid. The validity of all initial lane feature points is calculated by iterating through them to obtain the second lane feature point and its classification information. It is understandable that the deviation from the mean can also be used to calculate its validity. Specific methods are not limited here.

[0088] In this embodiment, the Gaussian mixture model can be trained offline and then applied online. The specific process is as follows: Figure 9 As shown:

[0089] In the offline portion, offline lane feature points and prior information about the offline road structure are acquired. Then, when it is determined that the offline lane feature points and the prior information about the offline road structure are correlated, the offline lane feature points and the prior information about the offline road structure are transformed into a road edge coordinate system, thus forming an offline dataset, such as... Figure 10 As shown; then, based on the spatial distribution of the offline lane line feature points and the prior information of the offline road structure, the Gaussian mixture model is constructed, and the model parameter values ​​of the Gaussian mixture model are solved, as shown. Figure 11 As shown. It is understandable that the EM algorithm can be used to solve for the model parameter values ​​of this Gaussian mixture model.

[0090] In the online portion, the lane line detection device acquires initial lane line feature points and prior road structure information in real time. Then, after determining that the initial lane feature points are correlated with the prior road structure information, it transforms the initial lane feature points into first lane line feature points in the road edge coordinate system. The first lane line feature points are then input into the Gaussian mixture model trained in the offline portion to classify and denoise the first lane line feature points to obtain second lane line feature points. The model parameter values ​​of the Gaussian mixture model are updated based on the output of the second lane line feature points. Finally, the final lane line is output based on the second lane line feature points.

[0091] Optionally, in the online portion, the lane detection device can also update the model's parameter values ​​(including the mean and variance of each sub-model of the Gaussian mixture model and the weights of the sub-models) based on the distribution of the effective lane feature points finally output by the Gaussian mixture model in the road edge coordinate system.

[0092] The point cloud feature map obtained using the above method can be as follows: Figure 12 As shown, it effectively suppresses the impact of noise from road signs (text, arrows) on lane line detection.

[0093] 505. The lane line detection device outputs the final lane line based on the feature points of the second lane line.

[0094] In this embodiment, the following can be used: Figure 13 The flowchart shown outputs the final lane line. Specifically, the lane line detection device uses a Bayesian classifier to obtain the effective lane line feature points from the second lane line feature points, based on the second lane line feature points and prior cues. The prior cues include, but are not limited to, confidence level, similarity between the second lane line feature points and historical tracking results, and the probability value of the absolute value of the reflection intensity of the second lane line feature points. The absolute value of the reflection intensity is used to indicate the relative change in point cloud reflectivity. Finally, a cost function is used to optimize the effective lane line feature points using a nonlinear optimization method to output the final lane line.

[0095] In this embodiment, the confidence level is obtained by the lane line detection device inputting the first lane line feature points into the Gaussian mixture model. The confidence level is used to indicate the probability value of the second lane line feature point as a valid feature point. That is, when the Gaussian mixture model outputs a valid feature point, it also outputs the probability value of that valid feature point. For example, feature points with a probability value greater than 0.6 in the first lane line feature points are used as the second lane line feature points. Therefore, the second lane line feature points also include feature points with a probability value of 0.7 and feature points with a probability value of 0.8.

[0096] The similarity between the second lane line feature points and historical tracking results can serve as a valid prior clue in lane line detection algorithms. In practice, vehicle pose data is used to transform historical lane line tracking results into the coordinate space of the current data frame. The lane line feature points in the current data frame are then correlated with historical tracking results. The distance from the feature points to the associated tracking results is calculated, and this distance is mapped to a probability space to provide a similarity score. Vehicle pose data refers to the data stream reflecting the vehicle's position and attitude obtained through a combined inertial navigation system (GPS / IMU).

[0097] The probability value of the absolute value of the reflection intensity of the second lane feature point can be obtained using the following technical solution: Since the extraction of the initial lane feature point depends on the relative change characteristics of the point cloud reflectivity intensity, the absolute value of the reflection intensity of the point cloud on the lane line is also in a high-intensity range. The range value is related to the performance of the LiDAR. The absolute value of the reflection intensity and the range are mapped to a probability space to give the probability value of the absolute value of the reflection intensity. For example: Assume the range of the LiDAR's reflection intensity for the lane line is [I... min ,I max The prior probability of a lane feature point in terms of reflection intensity can be expressed as follows:

[0098]

[0099] Among them, the P Intensity The probability value represents the absolute value of the reflection intensity of the feature point of the second lane line; the Intensity represents the absolute value of the reflection intensity of the feature point of the second lane line.

[0100] Based on the probability values ​​of the prior clues, the posterior probability of the second lane feature point is calculated using Bayesian inference, and a validity probability threshold is set. If the output posterior probability value is higher than the validity probability threshold, then the lane feature point corresponding to that posterior probability value is the final lane feature point. The specific formula is as follows:

[0101]

[0102] Wherein, P(valid) is the posterior probability value, P(GMM) is the confidence level, P(History) is the similarity between the second lane feature point and the historical tracking results, and P(Intensity) is the probability value of the absolute value of the reflection intensity of the second lane feature point.

[0103] In this embodiment, the cost function is designed according to preset rules; the preset rules include at least: 1. The spatial distribution of lane lines and the spatial distribution of road structure are consistent. In this embodiment, the consistency between the spatial distribution of lane lines and the spatial distribution of road structure can be reflected in the parallel similarity between the road curb and the lane line curve. 2. The lane line output of continuous frame data has continuity and consistency, that is, the fit between the lane line feature points and the parametric equation (used to represent the final lane line). In this embodiment, the fit between the lane line feature points and the parametric equation can be reflected in the average point-to-line distance from the lane line feature points to the parametric equation.

[0104] In the technical solution provided in this embodiment, the lane line detection device performs road prior judgment on the road structure prior information and the initial lane line feature points, thereby pre-screening the initial lane line feature points and the road structure prior information. Then, it establishes a road edge coordinate system using the road structure prior information and transforms the initial lane line feature points within this coordinate system, thus ensuring the validity of the initial lane line feature points and the road structure prior information. Finally, it inputs the data into a Gaussian mixture model for classification and denoising. Since the Gaussian mixture model can effectively learn the spatial distribution characteristics of lane lines of corresponding types of roads, it effectively classifies and denoises the lane line feature points, ultimately improving the accuracy of lane line detection. Simultaneously, it uses multiple prior clues to judge the validity of lane line feature points, essentially filtering out noise in the feature points. Furthermore, it designs a multi-constraint cost function and solves its optimal lane line parameter equation, improving the stability and accuracy of the lane line parameterized output.

[0105] The lane line detection method in the embodiments of this application has been described above. It is understood that the lane line detection device, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] This application embodiment can divide the lane line detection device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0107] The lane line detection device in the embodiments of this application is described in detail below. Please refer to the following for details. Figure 14 As shown, Figure 14 This is a schematic diagram of one embodiment of the lane line detection device 1400 in this application. The lane line detection device 1400 includes: an acquisition module 1401, used to acquire prior information about the road structure and initial lane line feature points;

[0108] Processing module 1402 is used to convert the initial lane line feature points into first lane line feature points in the road edge coordinate system when the correlation between the initial lane line feature points and the road structure prior information is determined to reach a preset threshold according to the road prior decision rule; input the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, wherein the Gaussian mixture model is used to classify and denoise the first lane line feature points; and output the final lane line based on the second lane line feature points.

[0109] Optionally, the road prior decision rule includes at least one of the following:

[0110] The length of the road structure exceeds the first preset value;

[0111] The endpoints of the curve of the road structure meet preset conditions. The preset conditions are that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is less than a second preset value, and the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is greater than a third preset value, wherein the second preset value is less than the third preset value.

[0112] The parallelism of the roadside curbs on both sides of the road structure is greater than the parallelism threshold;

[0113] The coupling degree between the road structure and the feature points of the side lane lines is greater than the coupling degree threshold.

[0114] Optionally, the processing module 1402 is further configured to input the first lane line feature points into the Gaussian mixture model to obtain confidence scores, wherein the confidence scores are used to indicate the probability values ​​of the second lane line feature points.

[0115] Optionally, the acquisition module 1401 is specifically used to acquire the original laser point cloud data and the prior information of the road structure based on laser sensing technology;

[0116] The processing module 1402 is specifically used to preprocess the original laser point cloud data to obtain the initial lane line feature points.

[0117] Optionally, the processing module 1402 is specifically used to obtain effective lane line feature points of the second lane line feature points by using a Bayesian classifier with the second lane line feature points and prior clues. The prior clues include, but are not limited to, the confidence score, the similarity between the second lane line feature points and historical tracking results, and the probability value of the absolute value of the reflection intensity of the second lane line feature points. The absolute value of the reflection intensity is used to indicate the relative change feature of the point cloud reflectivity intensity. The effective lane line feature points are optimized using a cost function and a nonlinear optimization method to output the final lane line.

[0118] Optionally, the cost function is designed according to preset rules;

[0119] The preset rules include at least the following:

[0120] The spatial distribution of lane markings is consistent with the spatial distribution of the road.

[0121] The lane line output of continuous frame data has continuity and consistency.

[0122] Optionally, the acquisition module 1401 is further configured to acquire an offline dataset;

[0123] The processing module 1402 is also used to train the Gaussian mixture model based on the offline dataset.

[0124] Optionally, the acquisition module 1401 is specifically used to acquire offline road structure prior information and offline lane line feature points;

[0125] The processing module 1402 is specifically used to convert the offline lane line feature points and the offline road structure prior information into the offline dataset in the road edge coordinate system when the correlation between the offline lane line feature points and the offline road structure prior information reaches a preset threshold according to the road prior decision rule.

[0126] Optionally, the processing module 1402 is specifically used to construct the Gaussian mixture model based on the spatial distribution of the offline lane line feature points and the prior information of the offline road structure, and to solve for the model parameter values ​​of the Gaussian mixture model.

[0127] Optionally, the processing module 1402 is further configured to update the model parameter values ​​based on the effective lane line feature points.

[0128] The lane line detection device in the above embodiments can be a smart car, or it can be a chip or other combined device or component that can realize the above-mentioned smart car functions. When the lane line detection device is a smart car, the transceiver module can be a transceiver, which can include an antenna and radio frequency circuits, etc., and the processing module can be a processor, such as a baseband chip. When the lane line detection device is a component with the above-mentioned lane line detection function, the transceiver module can be a radio frequency unit, and the processing module can be a processor. When the lane line detection device is a chip system, the part of the transceiver module used for receiving can be the input port of the chip system, the part of the transceiver module used for transmitting can be the output interface of the chip system, and the processing module can be the processor of the chip system, such as a central processing unit (CPU).

[0129] In this embodiment, the lane line detection device further includes a storage module or memory, which is mainly used to store software programs and data, such as the programs described in the above embodiments. The lane line detection device also has the following functions:

[0130] Transceiver, acquires prior information about the road structure and initial lane line feature points;

[0131] When the processor determines that the correlation between the initial lane line feature points and the prior information of the road structure reaches a preset threshold according to the road prior decision rules, it transforms the initial lane line feature points into first lane line feature points in the road edge coordinate system; inputs the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, the Gaussian mixture model is used to classify and denoise the first lane line feature points; and outputs the final lane line based on the second lane line feature points.

[0132] Optionally, the road prior decision rule includes at least one of the following:

[0133] The length of the road structure exceeds the first preset value;

[0134] The endpoints of the curve of the road structure meet preset conditions. The preset conditions are that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is less than a second preset value, and the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is greater than a third preset value, wherein the second preset value is less than the third preset value.

[0135] The parallelism of the roadside curbs on both sides of the road structure is greater than the parallelism threshold;

[0136] The coupling degree between the road structure and the feature points of the side lane lines is greater than the coupling degree threshold.

[0137] Optionally, the processor is further configured to input the first lane line feature points into the Gaussian mixture model to obtain confidence scores, wherein the confidence scores are used to indicate the probability values ​​of the second lane line feature points.

[0138] Optionally, the transceiver is specifically used to acquire raw laser point cloud data and prior information about the road structure based on laser sensing technology;

[0139] The processor is specifically used to preprocess the original laser point cloud data to obtain the initial lane line feature points.

[0140] Optionally, the processor is specifically configured to use a Bayesian classifier to obtain effective lane line feature points of the second lane line feature points and prior clues, wherein the prior clues include, but are not limited to, the confidence score, the similarity between the second lane line feature points and historical tracking results, and the probability value of the absolute value of the reflection intensity of the second lane line feature points, wherein the absolute value of the reflection intensity is used to indicate the relative change feature of the point cloud reflectivity intensity; and to optimize the effective lane line feature points using a cost function and a nonlinear optimization method to output the final lane line.

[0141] Optionally, the cost function is designed according to preset rules;

[0142] The preset rules include at least the following:

[0143] The spatial distribution of lane markings is consistent with the spatial distribution of the road.

[0144] The lane line output of continuous frame data has continuity and consistency.

[0145] Optionally, the transceiver is also used to acquire offline datasets;

[0146] The processor is also used to train the Gaussian mixture model based on the offline dataset.

[0147] Optionally, the transceiver is specifically used to acquire offline road structure prior information and offline lane line feature points;

[0148] The processor is specifically configured to convert the offline lane line feature points and the offline road structure prior information into the offline dataset in the road edge coordinate system when the correlation between the offline lane line feature points and the offline road structure prior information is determined to reach a preset threshold according to the road prior decision rule.

[0149] Optionally, the processor is specifically configured to construct the Gaussian mixture model based on the spatial distribution of the offline lane line feature points and the prior information of the offline road structure, and solve for the model parameter values ​​of the Gaussian mixture model.

[0150] Optionally, the processor is further configured to update the model parameter values ​​based on the effective lane line feature points.

[0151] This application embodiment also provides a processing device. The processing device includes a processor and an interface; the interface is used to acquire prior information about the road structure and initial lane line feature points; the processor is used to, when the correlation between the initial lane line feature points and the prior information about the road structure reaches a preset threshold according to a road prior decision rule, convert the initial lane line feature points into first lane line feature points in a road edge coordinate system; input the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, the Gaussian mixture model being used to classify and denoise the first lane line feature points; and output the final lane line based on the second lane line feature points.

[0152] That is, the processor is used to execute the data processing method of any of the above method embodiments.

[0153] It should be understood that the aforementioned processing device can be a chip. The processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. The memory can be integrated into the processor or located outside the processor and exist independently.

[0154] "Implemented in hardware" refers to implementing the functions of the aforementioned modules or units through hardware processing circuits that do not have program instruction processing capabilities. These hardware processing circuits can be composed of discrete hardware components or integrated circuits. To reduce power consumption and size, integrated circuits are typically used. Hardware processing circuits can include ASICs (application-specific integrated circuits) or PLDs (programmable logic devices); PLDs can further include FPGAs (field-programmable gate arrays), CPLDs (complex programmable logic devices), and so on. These hardware processing circuits can be a single packaged semiconductor chip (e.g., packaged as an ASIC); or they can be integrated with other circuits (e.g., CPUs, DSPs) and packaged into a single semiconductor chip. For example, multiple hardware circuits and a CPU can be formed on a silicon substrate and packaged into a single chip, also known as a SoC. Alternatively, circuits for implementing FPGA functions and a CPU can be formed on a silicon substrate and packaged into a single chip, also known as a SoPC (system on a programmable chip).

[0155] This application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to control a data processing device to perform any of the implementations shown in the foregoing method embodiments.

[0156] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the implementation methods shown in the foregoing method embodiments.

[0157] This application also provides a chip system, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the chip performs any of the implementation methods shown in the foregoing method embodiments.

[0158] This application also provides a chip system including a processor, which is used to call and run a computer program, causing the chip to execute any of the implementation methods shown in the foregoing method embodiments.

[0159] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device to execute the methods described in the various embodiments of this application.

[0161] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0162] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, first network device or second network device, computing device, or data center to another website, computer, first network device or second network device, computing device, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a first network device or second network device, data center, etc., that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0163] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence number of the above-described processes does not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0164] Furthermore, the terms "system" and "network" are often used interchangeably in this document. It should be understood that, in the embodiments of this application, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0165] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0166] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0170] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

Claims

1. A method for detecting lane lines, characterized in that, include: Prior information about the road structure and initial lane line feature points are obtained using laser sensing technology; When the correlation between the initial lane line feature point and the prior information of the road structure reaches a preset threshold according to the road prior decision rule, the initial lane line feature point is transformed into the first lane line feature point in the road edge coordinate system. The road prior decision rule includes: the endpoint position of the curve of the road structure meets the preset conditions, the preset conditions are that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is less than a second preset value, the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is greater than a third preset value, and the second preset value is less than the third preset value. The first lane feature points are input into a Gaussian mixture model to obtain the second lane feature points. The Gaussian mixture model is used to classify and denoise the first lane feature points. The final lane line is output based on the feature points of the second lane line.

2. The method according to claim 1, characterized in that, The road prior decision rule also includes at least one of the following: The length of the road structure exceeds the first preset value; The parallelism of the roadside curbs on both sides of the road structure is greater than the parallelism threshold; The coupling degree between the road structure and the feature points of the side lane lines is greater than the coupling degree threshold.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The confidence level is obtained by inputting the first lane feature point into the Gaussian mixture model, and the confidence level is used to indicate the probability value of the second lane feature point.

4. The method according to any one of claims 1 to 2, characterized in that, The acquisition of prior information on road structure and initial lane line feature points based on laser sensing technology includes: The original laser point cloud data and the prior information of the road structure are obtained based on laser sensing technology; The initial lane line feature points are obtained by preprocessing the original laser point cloud data.

5. The method according to claim 4, characterized in that, The step of outputting the final lane line based on the second lane line feature points includes: The second lane feature point and prior clues are used to obtain the effective lane feature point of the second lane feature point by a Bayes classifier. The prior clues include the confidence score, the similarity between the second lane feature point and the historical tracking results, and the probability value of the absolute value of the reflection intensity of the second lane feature point. The absolute value of the reflection intensity is used to indicate the relative change feature of the point cloud reflectivity intensity. The final lane line is output by optimizing the effective lane line feature points using a cost function and a nonlinear optimization method.

6. The method according to claim 5, characterized in that, The cost function is designed according to preset rules; The preset rules include at least the following: The spatial distribution of lane markings is consistent with the spatial distribution of road structure. The lane line output of continuous frame data has continuity and consistency.

7. The method according to any one of claims 1 to 2, characterized in that, The method further includes: Obtain offline datasets; The Gaussian mixture model is trained based on the offline dataset.

8. The method according to claim 7, characterized in that, The acquisition of offline datasets includes: Acquire offline road structure prior information and offline lane line feature points; When the correlation between the offline lane line feature points and the offline road structure prior information reaches a preset threshold according to the road prior decision rule, the offline lane line feature points and the offline road structure prior information are transformed into the offline dataset in the road edge coordinate system.

9. The method according to claim 8, characterized in that, The step of training the Gaussian mixture model based on the offline dataset includes: The Gaussian mixture model is constructed based on the spatial distribution of the offline lane line feature points and the prior information of the offline road structure, and the model parameter values ​​of the Gaussian mixture model are obtained by solving.

10. The method according to claim 9, characterized in that, The method further includes: The model parameter values ​​are updated based on the effective lane line feature points.

11. A lane line detection device, characterized in that, include: The acquisition module is used to acquire prior information about the road structure and initial lane line feature points based on laser sensing technology; The processing module is used to convert the initial lane line feature points into first lane line feature points in the road edge coordinate system when the correlation between the initial lane line feature points and the road structure prior information reaches a preset threshold according to the road prior decision rules; input the first lane line feature points into a Gaussian mixture model to obtain second lane line feature points, wherein the Gaussian mixture model is used to classify and denoise the first lane line feature points; and output the final lane line based on the second lane line feature points. The road prior decision rule includes: the endpoint position of the curve of the road structure meets the preset conditions, the preset conditions are that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is less than a second preset value, the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle coordinate system is greater than a third preset value, and the second preset value is less than the third preset value.

12. The apparatus according to claim 11, characterized in that, The road prior decision rule also includes at least one of the following: The length of the road structure exceeds the first preset value; The parallelism of the roadside curbs on both sides of the road structure is greater than the parallelism threshold; The coupling degree between the road structure and the feature points of the side lane lines is greater than the coupling degree threshold.

13. The apparatus according to claim 11 or 12, characterized in that, The processing module is further configured to input the first lane line feature points into the Gaussian mixture model to obtain confidence scores, wherein the confidence scores are used to indicate the probability values ​​of the second lane line feature points.

14. The apparatus according to any one of claims 11 to 12, characterized in that, The acquisition module is specifically used to acquire raw laser point cloud data and the prior information of the road structure based on laser sensing technology. The processing module is specifically used to preprocess the original laser point cloud data to obtain the initial lane line feature points.

15. The apparatus according to claim 14, characterized in that, The processing module is specifically used to obtain effective lane line feature points of the second lane line feature points by using a Bayesian classifier with the second lane line feature points and prior clues. The prior clues include the confidence score, the similarity between the second lane line feature points and historical tracking results, and the probability value of the absolute value of the reflection intensity of the second lane line feature points. The absolute value of the reflection intensity is used to indicate the relative change feature of the point cloud reflectivity intensity. The effective lane line feature points are optimized using a cost function and a nonlinear optimization method to output the final lane line.

16. The apparatus according to claim 15, characterized in that, The cost function is designed according to preset rules; The preset rules include at least the following: The spatial distribution of lane markings is consistent with the spatial distribution of the road. The lane line output of continuous frame data has continuity and consistency.

17. The apparatus according to any one of claims 11 to 12, characterized in that, The acquisition module is also used to acquire offline datasets; The processing module is also used to train the Gaussian mixture model based on the offline dataset.

18. The apparatus according to claim 17, characterized in that, The acquisition module is specifically used to acquire offline road structure prior information and offline lane line feature points; The processing module is specifically used to convert the offline lane line feature points and the offline road structure prior information into the offline dataset in the road edge coordinate system when the correlation between the offline lane line feature points and the offline road structure prior information reaches a preset threshold according to the road prior decision rule.

19. The apparatus according to claim 18, characterized in that, The processing module is specifically used to construct the Gaussian mixture model based on the spatial distribution of the offline lane line feature points and the prior information of the offline road structure, and to solve for the model parameter values ​​of the Gaussian mixture model.

20. The apparatus according to claim 19, characterized in that, The processing module is also used to update the model parameter values ​​based on the effective lane line feature points.

21. A lane line detection device, characterized in that, It includes at least one processor and a memory, the processor being coupled to the memory, the processor invoking instructions stored in the memory to control the data processing device to perform the method of any one of claims 1 to 10.

22. A computer storage medium storing computer instructions for performing the method described in any one of claims 1 to 10.

23. A computer program product comprising instructions that, when run on a computer, causes the computer to perform the method described in any one of claims 1 to 10.

24. An intelligent vehicle, characterized in that, The intelligent vehicle includes the device according to any one of claims 11 to 20.

Images