Lane line detection method and apparatus, and device and medium

Abstract

A lane line detection method and apparatus, and a device and a medium. The method comprises: acquiring three-dimensional radar point cloud data of a traveling direction environment of a vehicle (S110); performing grid division on the three-dimensional radar point cloud data, so as to obtain grid feature data (S120); inputting the grid feature data into a deep polynomial regression model, so as to obtain at least one set of output data (S130), wherein each set of output data comprises polynomial parameters in a preset univariate polynomial function and first coordinate data of a lane line target point, and the first coordinate data is coordinate data of the lane line target point in a traveling direction; for each set of output data, obtaining a target polynomial function on the basis of the polynomial parameters and the preset univariate polynomial function in the output data (S140); and on the basis of the target polynomial function, and the first coordinate data of the lane line target point in the output data, obtaining a target lane line corresponding to each set of output data (S150).

Description

Lane detection methods, devices, equipment and media

[0001] This application claims priority to Chinese Patent Application No. 202410803410.3, filed with the Chinese Patent Office on June 20, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of autonomous driving technology, such as a lane line detection method, device, equipment, and medium. Background Technology

[0003] Autonomous driving cannot function without the detection of road boundaries. Road boundary detection is a core functional module in driver assistance systems that provides a safe driving area for humans, and it is also an important prerequisite functional module that provides an effective detection range and driving area for fully autonomous driving.

[0004] Currently, there are methods for lane line detection based on radar point clouds. These methods typically use neural network models to segment the radar point clouds into lane lines, obtain point cloud segmentation results, and then perform post-processing on the point cloud segmentation results to obtain lane lines with curve fitting.

[0005] However, lane line detection methods based on radar point clouds in related technologies cannot directly obtain lane lines through neural network models and require complex post-processing. Summary of the Invention

[0006] This application provides a lane line detection method, apparatus, equipment, and medium to solve the problem of low lane line detection efficiency and improve lane line calculation speed.

[0007] According to one aspect of this application, a lane line detection method is provided, applied to a vehicle equipped with onboard radar, the method comprising:

[0008] Acquire 3D radar point cloud data of the vehicle's driving direction environment;

[0009] The 3D radar point cloud data is divided into grids to obtain grid feature data;

[0010] The raster feature data is input into the deep multinomial regression model to obtain at least one set of output data; each set of output data includes the polynomial parameters in the preset univariate multinomial function and the first coordinate data of the lane line target point; the first coordinate data is the coordinate data of the lane line target point in the driving direction;

[0011] For each set of output data, a target polynomial function is obtained based on the polynomial parameters in the output data and the preset univariate multinomial function; the target lane line corresponding to each set of output data is obtained based on the target polynomial function and the first coordinate data of the lane line target point in the output data.

[0012] According to another aspect of this application, a lane detection device is provided, configured in a vehicle equipped with onboard radar, comprising:

[0013] The 3D radar point cloud data acquisition module is configured to acquire 3D radar point cloud data of the vehicle's driving direction environment.

[0014] The raster feature data determination module is configured to perform raster division on 3D radar point cloud data to obtain raster feature data.

[0015] The output data determination module is configured to input raster feature data into a deep multinomial regression model to obtain at least one set of output data; each set of output data includes polynomial parameters in a preset univariate multinomial function and the first coordinate data of the lane line target point; the first coordinate data is the coordinate data of the lane line target point in the driving direction;

[0016] The target lane line determination module is configured to, for each set of output data, obtain a target polynomial function based on the polynomial parameters in the output data and a preset univariate multinomial polynomial function; and obtain the target lane line corresponding to each set of output data based on the target polynomial function and the first coordinate data of the lane line target point in the output data.

[0017] According to another aspect of this application, an electronic device is provided, comprising:

[0018] At least one processor; and

[0019] A memory that is communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the lane detection method of any embodiment of this application.

[0021] According to another aspect of this application, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the lane detection method of any embodiment of this application. Attached Figure Description

[0022] Figure 1 is a flowchart of a lane line detection method provided in Embodiment 1 of this application;

[0023] Figure 2 is a two-dimensional visualization of the three-dimensional radar point cloud data of Embodiment 1 of this application;

[0024] Figure 3 is a two-dimensional visualization of the target lane line in Embodiment 1 of this application;

[0025] Figure 4 is a flowchart of a lane line detection method provided in Embodiment 2 of this application;

[0026] Figure 5 is a structural schematic diagram of a lane line detection device provided in Embodiment 3 of this application;

[0027] Figure 6 is a schematic diagram of the structure of an electronic device that implements the lane line detection method of the present application. Detailed Implementation

[0028] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, including processes, methods, systems, products, or devices that, in addition to including the series of steps or units shown in the embodiments of this application, may also include processes, methods, systems, products, or devices that do not list such series of steps or units, or other steps or units inherent to such processes, methods, systems, products, or devices.

[0029] Example 1

[0030] Figure 1 is a flowchart of a lane line detection method provided in Embodiment 1 of this application. This embodiment is applicable to the extraction of lane lines based on radar point clouds. This method can be applied to vehicles equipped with onboard radar. The method is executed by a lane line detection device, which can be implemented in hardware and / or software and can be configured in lane line detection. As shown in Figure 1, the method includes:

[0031] S110. Acquire three-dimensional radar point cloud data of the vehicle's driving direction environment.

[0032] In this embodiment, the vehicle is equipped with onboard radar, which scans the environment along the vehicle's driving direction to obtain three-dimensional radar point cloud data. The onboard radar includes lidar, ultrasonic radar, and millimeter-wave radar, among others. The vehicle's driving direction refers to the direction the vehicle is currently traveling, which can be understood as including four basic directions: forward, backward, left turn, and right turn. The environment along the vehicle's driving direction includes, for example, the road environment, road surface conditions, traffic conditions, and road obstacles. The three-dimensional radar point cloud data is three-dimensional point cloud data obtained based on the onboard radar scanning the environment along the driving direction.

[0033] For example, during vehicle operation, the external environment is scanned in real time using onboard radar. The scanning results are then read to obtain three-dimensional point cloud data of the external environment. Based on this three-dimensional point cloud data, three-dimensional radar point cloud data for the driving direction environment is extracted. In some embodiments, the external environment can be the environment surrounding the vehicle, including the driving direction environment. The three-dimensional radar point cloud data for the driving direction environment is obtained by extracting the three-dimensional point cloud data of the driving direction environment from the three-dimensional point cloud data of the external environment. In some embodiments, if the external environment is only the driving direction environment, then the three-dimensional point cloud data of the external environment is directly used as the three-dimensional radar point cloud data for the driving direction environment.

[0034] For example, Figure 2 is a two-dimensional visualization of the three-dimensional radar point cloud data provided in Embodiment 1 of this application. As shown in Figure 2, there is a clear road edge outline in the three-dimensional radar point cloud data.

[0035] 3D radar point cloud data may contain invalid points due to vehicle radar errors, environmental interference, or data corruption. Furthermore, the coordinate system of the 3D radar point cloud data obtained by reading the vehicle radar scan results is the radar coordinate system, while the coordinate system of the 3D radar point cloud data in subsequent processes should be the vehicle body coordinate system. Optionally, the 3D radar point cloud data can be preprocessed; this preprocessing includes removing invalid points and converting the coordinate system of the preprocessed 3D radar point cloud data to the vehicle body coordinate system.

[0036] For example, the data quality of 3D radar point cloud data can be improved by removing invalid points, where invalid points include non-numerical (NaN) points and infinity (Inf) points. For example, invalid points can be removed from the 3D radar point cloud data by calling a point cloud processing application or point cloud processing tool library.

[0037] A vehicle coordinate system is constructed with the vehicle's key points as the origin, the vehicle's direction of travel as the positive X-axis, the direction perpendicular to the vehicle's bottom (e.g., clockwise or counterclockwise) as the positive Y-axis, and the direction vertically upwards from the vehicle's bottom as the positive Z-axis. For example, the vehicle's key points can be the vehicle's center point or specific location points. A rotation matrix is ​​determined based on the installation location of the onboard radar and the vehicle's direction of travel to transform the radar coordinate system to the vehicle coordinate system. The preprocessed 3D radar point cloud data is then transformed from the radar coordinate system to the vehicle coordinate system based on this rotation matrix.

[0038] This embodiment improves the data quality of 3D radar point cloud data by preprocessing it, which helps to improve the accuracy of subsequent lane line detection.

[0039] S120. Divide the 3D radar point cloud data into grids to obtain grid feature data.

[0040] In this embodiment, the raster feature data is data in two-dimensional space, used to characterize the features of three-dimensional radar point cloud data in three-dimensional space.

[0041] For example, at least a local portion of the 3D radar point cloud data corresponding to the target projection area is determined as the 3D radar point cloud data to be segmented. The target projection area is at least a local region of the point cloud projection area, which is the downward projection area of ​​the 3D radar point cloud data in the vertical direction (negative Z-axis) of the vehicle's travel direction (X-axis). Based on a preset interval size, the target projection area is divided into multiple target projection sub-regions. The preset interval size is used to equally divide the target projection area, and it is determined based on the computing power of the computing device used for lane detection. It is understood that the smaller the preset interval size, the greater the computing power requirement of the computing device.

[0042] For example, the target projection area is 64 meters × 160 meters, wherein the target projection area is 160 meters in the vehicle driving direction and 64 meters in the vertical direction parallel to the bottom of the vehicle in the vehicle driving direction. The preset interval size is 0.2 meters × 0.2 meters. Then the target projection area includes 320 × 800 target projection sub-areas.

[0043] The 3D radar point cloud data to be divided is based on multiple target projection sub-regions, resulting in grid point cloud data corresponding to each target projection sub-region. The downward projection area of ​​each target projection sub-region in the vertical direction of vehicle travel is defined as that target projection sub-region. Feature extraction and dimensionality reduction to two-dimensional space are performed on the grid point cloud data corresponding to each of the multiple target projection sub-regions to obtain grid feature data. For example, assuming the target projection area includes M×N target projection sub-regions, the grid feature data includes M×N data elements.

[0044] In some embodiments, the three-dimensional radar point cloud data includes X-axis coordinate information, Y-axis coordinate information, Z-axis coordinate information, and intensity information corresponding to multiple target points. In this embodiment, a target point is a point in the three-dimensional radar point cloud data. The target point is a point in three-dimensional space. The X-axis coordinate information represents its position along the X-axis of the vehicle coordinate system, the Y-axis coordinate information represents its position along the Y-axis of the vehicle coordinate system, and the Z-axis coordinate information represents its position along the Z-axis of the vehicle coordinate system. For example, the p-th target point is represented as O. p =(x p ,y p ,z p i p ), where x p Represents X-axis coordinate information, y p Represents the Y-axis coordinate information, z p Represents Z-axis coordinate information, i p This represents intensity information. It can be understood that the raster feature data is obtained by dimensionality reduction of the Z-axis dimension of the 3D radar point cloud data. Optionally, based on the X-axis and Y-axis coordinates of each target point, the cell raster to which each target point belongs is determined; based on the Z-axis coordinates and intensity information of at least one target point contained in the cell raster, and the X-axis and Y-axis coordinates of the cell raster, the cell raster feature data is determined; based on the cell raster feature data corresponding to multiple cell rasteres, the raster feature data is determined.

[0045] In this embodiment, the grid corresponds to the target projection area, and the X-axis coordinate range (or Y-axis coordinate range) of the grid is the X-axis coordinate range (or Y-axis coordinate range) of the target projection area. It can be understood that the grid comprises multiple meshes, and a cell grid is one mesh within the grid. Each cell grid corresponds to one target projection sub-region. For example, assuming the target projection area comprises M×N target projection sub-regions, the grid comprises M×N cell grids, where the index of the cell grid in the m-th row and n-th column is mn, where m∈[1,M] and n∈[1,N].

[0046] For example, for each cell grid, if the X-axis coordinate information corresponding to any target point is within the X-axis coordinate range of the cell grid, and the Y-axis coordinate information corresponding to the target point is within the Y-axis coordinate range of the cell grid, then the target point is determined to belong to that cell grid. If any cell grid contains zero target points, the cell grid feature data corresponding to that cell grid is set to zero. If any cell grid contains at least one target point, the Z-axis feature information corresponding to that cell grid is obtained by extracting Z-axis coordinate features from the Z-axis coordinate information corresponding to each of the at least one target point, and the intensity feature information corresponding to that cell grid is obtained by extracting intensity features from the intensity information corresponding to each of the at least one target point. For example, the Z-axis feature information or intensity feature information can be obtained by feature extraction based on neural networks or traditional feature extraction methods. Optionally, the maximum, minimum, and average Z-axis coordinate information among the Z-axis coordinate information corresponding to each of the at least one target point contained in the cell grid are determined; the maximum, minimum, and average intensity information among the intensity information corresponding to each of the at least one target point contained in the cell grid are also determined.

[0047] For example, for each cell grid, statistical analysis is performed on the Z-axis coordinate information corresponding to at least one target point. The largest Z-axis coordinate is determined as the maximum Z-axis coordinate, the smallest Z-axis coordinate is determined as the minimum Z-axis coordinate, and the average Z-axis coordinate is determined as the average Z-axis coordinate. The methods for determining the maximum intensity information, minimum intensity information, and average intensity information are similar. For example, the Z-axis feature information Fz of the cell grid feature data corresponding to the cell grid in the m-th row and n-th column is... mn Includes maximum Z-axis coordinate information z mn,max Minimum Z-axis coordinate information z mn,min and average Z-axis coordinate information z mn,mean Intensity feature information Fi mn Including maximum intensity information i mn,max Minimum intensity information i mn,min and average intensity information i mn,mean .

[0048] The grid X-axis coordinate information represents the position of a cell grid along the X-axis, and the grid Y-axis coordinate information represents the position of a cell grid along the Y-axis. In some embodiments, the grid X-axis coordinate information or grid Y-axis coordinate information can be determined based on the index of a cell grid among multiple cell grids and the size of the cell grid. Taking the grid X-axis coordinate information as an example, assuming that the index of a cell grid is mn and the size of the cell grid is k1 meters × k2 meters, then the grid X-axis coordinate information X of this cell grid is... mnThe value is k1m / 2 meters. In some embodiments, optionally, the grid X-axis coordinate information corresponding to the unit grid is determined based on the X-axis coordinate range corresponding to the unit grid, and the grid Y-axis coordinate information corresponding to the unit grid is determined based on the Y-axis coordinate range corresponding to the unit grid.

[0049] For example, the X-axis coordinates of marker points are calculated based on the maximum and minimum X-axis coordinates within the X-axis coordinate range corresponding to a cell grid. The resulting X-axis coordinates of the marker points are then determined as the X-axis coordinates of the cell grid. The marker points may include, for example, the center point and one-third point of the X-axis coordinate range corresponding to the cell grid. The method for determining the X-axis coordinates of the grid is similar to the method for determining the X-axis coordinates of the cell grid. For example, assuming the marker point is the center point of the X-axis coordinate range corresponding to the cell grid, the X-axis coordinate range corresponding to the cell grid in the m-th row and n-th column is [x...]. mn,min ,x mn,max ], then the grid X-axis coordinate information of this unit grid. mn For (x) mn,min +x mn,max ) / 2.

[0050] By integrating the Z-axis feature information, intensity feature information, X-axis coordinate information, and Y-axis coordinate information corresponding to the unit grid, the unit grid feature data corresponding to the unit grid is obtained. Among them, the Z-axis feature information is data characterizing the distribution characteristics of at least one target point in the unit grid along the Z-axis, and the intensity feature information is data characterizing the intensity distribution characteristics of at least one target point in the unit grid. For example, the unit grid feature data F corresponding to the unit grid in the m-th row and n-th column is... mn =(X mn ,Y mn ,Fz mn ,Fi mn ), where Y mn The grid's Y-axis coordinate information is represented. Optionally, the maximum Z-axis coordinate information, minimum Z-axis coordinate information, average Z-axis coordinate information, maximum intensity information, minimum intensity information, average intensity information, grid X-axis coordinate information, and grid Y-axis coordinate information are determined as the cell grid feature data corresponding to the cell grid. In this embodiment, the cell grid feature data includes information in 8 dimensions. For example, the cell grid feature data corresponding to the cell grid in the m-th row and n-th column is represented as F. mn =(X mn ,Y mn ,z mn,max ,z mn,min ,z mn,mean i mn,max i mn,min i mn,mean ).

[0051] This embodiment determines the maximum Z-axis coordinate information, minimum Z-axis coordinate information, average Z-axis coordinate information, maximum intensity information, minimum intensity information, average intensity information, grid X-axis coordinate information, and grid Y-axis coordinate information as the unit grid feature data corresponding to the unit grid. Among them, the maximum Z-axis coordinate information, minimum Z-axis coordinate information, and average Z-axis coordinate information (or maximum intensity information, minimum intensity information, and average intensity information) can provide extreme values, boundary values, and overall trends in the Z-axis coordinate information (or intensity information) of at least one target point contained in the unit grid, which helps to improve the accuracy of subsequent lane line detection.

[0052] The raster feature data is determined based on the feature data corresponding to multiple raster cells. For example, the raster feature data is obtained by integrating the feature data corresponding to multiple raster cells based on the raster cell index. For example, assume the raster feature data includes M×N data elements, where the data element in the m-th row and n-th column is the raster cell feature data F. mn .

[0053] This embodiment determines the cell feature data corresponding to a cell grid by using the Z-axis coordinate information and intensity information corresponding to at least one target point contained in the cell grid, as well as the X-axis coordinate information and Y-axis coordinate information of the cell grid. This allows the cell feature data determined based on the cell grid feature data to compress information while maintaining the original information types compared to the 3D radar point cloud data, which helps to ensure the accuracy of lane line detection.

[0054] S130. Input the raster feature data into the deep multinomial regression model to obtain at least one set of output data.

[0055] Each set of output data includes the polynomial parameters in a preset univariate multinomial function and the first coordinate data of the lane line target point; the first coordinate data is the coordinate data of the lane line target point in the driving direction.

[0056] In this embodiment, the deep multinomial regression model is a regression model built on a neural network, used to automatically map raster feature data into at least one set of output data.

[0057] For example, a deep multinomial regression model is used to predict the input raster feature data, resulting in at least one set of output data. The number of output data sets is less than or equal to a preset threshold, which controls the number of output data sets from the deep multinomial regression model. This preset threshold is set based on the maximum number of lane lines in a real lane. For example, if the preset threshold is 6, then the number of output data sets from the deep multinomial regression model will be less than or equal to 6.

[0058] Each set of output data corresponds to a lane line. The preset univariate polynomial function is a pre-defined polynomial function, including multiple power terms, the number of which is the same as the degree of the preset univariate polynomial function. The lane line target point is the point on the lane line corresponding to the output data. It can be understood that the lane line target point includes at least the lane line target point corresponding to the first coordinate data included in the output data, wherein the lane line target point corresponding to the first coordinate data included in the output data includes at least the lane line start point and lane line end point. The polynomial parameters include the coefficients of each power term in the preset univariate polynomial function. For example, assuming the preset univariate polynomial function is a univariate W-degree polynomial function, the polynomial parameters include the polynomial parameters corresponding to W+1 power terms. Optionally, the univariate multinomial function is preset to a univariate cubic polynomial function; and / or, the deep polynomial regression model includes a backbone network, a feature straightening layer, and a fully connected layer; wherein, the backbone network is used to extract lane line features from the raster feature data; the feature straightening layer is used to convert the lane line features into one-dimensional lane line features; and the fully connected layer is used to predict at least one set of output data based on the one-dimensional lane line features.

[0059] In some embodiments, the preset univariate polynomial function can be a univariate cubic polynomial function, in which case the polynomial parameters include the coefficients of four power terms. The univariate cubic polynomial function can be used to draw straight lines, curves, and complex curves. By setting the preset univariate polynomial function to a univariate cubic polynomial function, computational resources can be saved while ensuring the accuracy of lane detection.

[0060] In some embodiments, the deep multinomial regression model includes a backbone network, a feature straightening layer, and a fully connected layer. The model structure of the deep multinomial regression model includes a backbone network, a feature straightening layer, and a fully connected layer connected sequentially. The raster feature data input to the deep multinomial regression model is processed by the backbone network for feature extraction to obtain lane line features; the lane line features are straightened into one-dimensional vectors by the feature straightening layer to obtain one-dimensional lane line features; the one-dimensional lane line features are then processed by the fully connected layer for output data prediction to obtain at least one set of output data. For example, the backbone network is a Residual Network-18 (ResNet-18). By constructing a deep multinomial regression model based on a backbone network, a feature straightening layer, and a fully connected layer, the deep multinomial regression model can leverage the advantages of deep learning and multinomial regression to improve the reliability of the output data, thus contributing to improved lane line detection accuracy.

[0061] In some embodiments, the preset univariate multinomial function can be a univariate cubic polynomial function, in which case the polynomial parameters include the coefficients of four power terms. The deep polynomial regression model includes a backbone network, a feature straightening layer, and a fully connected layer, which can save computational resources and improve lane line detection accuracy.

[0062] In other embodiments, the maximum degree of the predefined univariate multivariate polynomial function and the model structure of the deep polynomial regression model can be set according to actual needs.

[0063] S140. For each set of output data, a target polynomial function is obtained based on the polynomial parameters in the output data and the preset univariate polynomial function.

[0064] In this embodiment, the target polynomial function is a polynomial function that characterizes the lane lines corresponding to a set of output data.

[0065] For example, for each set of output data, the coefficients of each power term in a preset univariate polynomial function are set based on the polynomial parameters in the output data to obtain the target polynomial function corresponding to the output data.

[0066] For example, suppose the preset univariate polynomial function is a univariate polynomial function of degree W, and the polynomial parameters in a set of output data include a W ,…a w ..., a1, a0, where a w Let a be the coefficient of the power term w, which is determined by setting the coefficient of the power term w in the preset univariate polynomial function to a. w , where w∈[0,W], yields the objective polynomial function representing the lane line corresponding to the output data.

[0067] S150. Based on the objective polynomial function and the first coordinate data of the lane line target point in the output data, obtain the target lane line corresponding to each set of output data.

[0068] In this embodiment, the target lane line is the lane line obtained by lane line detection of three-dimensional radar point cloud data, and each target lane line corresponds to a set of output data.

[0069] For example, for each set of output data, the coordinate data of each lane line target point is determined by inputting the first coordinate data of each lane line target point into the target polynomial function corresponding to the output data. Based on the coordinate data of multiple lane line target points, any two adjacent lane line target points with the first coordinate data are connected by a curve, and the drawn curve is determined as the target lane line.

[0070] For example, Figure 3 is a two-dimensional visualization of the target lane line in Embodiment 1 of this application. As shown in Figure 3, the white lines located on both sides of the road are the target lane lines. The lane line height represented by different target lane lines may be the same or different.

[0071] This embodiment inputs raster feature data into a deep polynomial regression model to obtain at least one set of output data. Based on the polynomial parameters in the output data and a preset univariate polynomial function, a target polynomial function is obtained. Based on the target polynomial function and the first coordinate data of the lane line target points in the output data, the target lane line corresponding to each set of output data is obtained. It eliminates the need to filter the points corresponding to the lane line based on confidence and perform curve fitting on the points corresponding to the lane line to obtain the target lane line. It can directly obtain the target lane line based on the output data and the preset univariate polynomial function, solving the problem of low lane line detection efficiency, improving lane line calculation speed, and saving computing resources.

[0072] Example 2

[0073] Figure 4 is a flowchart of a lane line detection method provided in Embodiment 2 of this application. This embodiment is an improvement on the above embodiments. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0074] In this embodiment, the lane line target point includes the lane line start point and the lane line end point; the target polynomial function is used to characterize the relationship between the second coordinate data of the lane line target point and the first coordinate data of the lane line target point; the second coordinate data is the coordinate data of the lane line target point in the vertical direction (Y-axis) of the driving direction.

[0075] For example, the lane line target point is a point on a plane parallel to the bottom of the vehicle in the direction of travel. The independent variable in the objective polynomial function is the first coordinate data, and the dependent variable is the second coordinate data. By inputting the first coordinate data of the lane line target point into the objective polynomial function, the second coordinate data corresponding to the first coordinate data is obtained.

[0076] For example, the objective polynomial function is y = a W ×x W +…+a w ×x w +…+a1×x+a0, where x is the first coordinate data, y is the second coordinate data, and a…+a1×x+a0… W ,…a w ,…,a1,a0 are polynomial parameters in a set of output data.

[0077] As shown in Figure 4, the method includes:

[0078] S210. Acquire three-dimensional radar point cloud data of the vehicle's driving direction environment.

[0079] S220. The three-dimensional radar point cloud data is divided into grids to obtain grid feature data.

[0080] S230. Input the raster feature data into the deep polynomial regression model to obtain at least one set of output data; each set of output data includes the polynomial parameters in the preset univariate multinomial function and the first coordinate data of the lane line target point; the first coordinate data is the coordinate data of the lane line target point in the driving direction.

[0081] S240. For each set of output data, a target polynomial function is obtained based on the polynomial parameters in the output data and the preset univariate polynomial function.

[0082] S250. Based on the first coordinate data of the starting point of the lane line and the first coordinate data of the ending point of the lane line, determine the range of the first coordinate data corresponding to the output data.

[0083] In this embodiment, the range of the first coordinate data is from the first coordinate data of the starting point of the lane line to the first coordinate data of the ending point of the lane line.

[0084] For example, the range of first coordinate data corresponding to the output data is determined by taking the first coordinate data of the starting point of the lane line as the lower bound and the first coordinate data of the ending point of the lane line as the upper bound.

[0085] For example, suppose the first coordinate data of the starting point of the lane line is x1, and the first coordinate data of the ending point of the lane line is x C The range of the first coordinate data corresponding to the output data is [x1, x...]. C ].

[0086] S260. Sample within the first coordinate data range to obtain multiple first sampled coordinate data. Input each first sampled coordinate data into the target polynomial function to obtain the second coordinate data corresponding to each first sampled coordinate data. Each first sampled coordinate data and the second coordinate data corresponding to the first sampled coordinate data form the coordinate data of a lane line target point.

[0087] In this embodiment, the first sampled coordinate data is the first coordinate data of any lane line target point between the lane line start point and the lane line end point. Multiple first sampled coordinate data are obtained by sampling within the first coordinate data range based on a preset sampling strategy. For example, the preset sampling strategy may include random sampling and equally spaced sampling. Optionally, multiple first sampled coordinate data are obtained by sampling the first coordinate data range corresponding to the output data at preset coordinate intervals.

[0088] In this embodiment, the preset coordinate interval is a pre-set sampling interval, which is determined based on the accuracy of the target lane line. It can be understood that the higher the accuracy of the target lane line, the smaller the preset coordinate interval. For example, assuming the number of the first sampled coordinate data is C, then the preset coordinate interval is x = (x... C -x1) / (C-1).

[0089] For example, for each range of first coordinate data corresponding to each output data point, the first coordinate data from the starting point of the lane line to the ending point of the lane line is sampled at equal intervals with a preset coordinate interval to obtain multiple first sampled coordinate data. It should be noted that the first sampled coordinate data includes the first coordinate data of the starting point of the lane line and the first coordinate data of the ending point of the lane line.

[0090] For example, assuming the first coordinate data range is [0, 800] and the preset coordinate interval is 2, it can be understood that the first coordinate data range corresponds to the range from 0 meters to 160 meters in the vehicle's driving direction, and the preset coordinate interval corresponds to a distance interval of 0.4 meters. Then, the c-th first sampled coordinate data x c It is 2×c, where c∈[0,400].

[0091] This embodiment can quickly obtain multiple first sampled coordinate data by sampling the range of first coordinate data corresponding to the output data at preset coordinate intervals, which helps to improve the efficiency of lane line detection.

[0092] For example, for each first sampled coordinate data, the first sampled coordinate data is input into a target polynomial function, causing the target polynomial function to perform polynomial calculations based on the first sampled coordinate data, thereby obtaining the second coordinate data corresponding to the first sampled coordinate data. Based on each first sampled coordinate data and the corresponding second coordinate data, a lane line target point is uniquely determined, and the coordinate data of the lane line target point includes the first sampled coordinate data and the corresponding second coordinate data.

[0093] For example, suppose the c-th first sampled coordinate data x c The second coordinate data corresponding to the c-th first sampled coordinate data is y. c Then the coordinate data of the lane line target point corresponding to the c-th first sampled coordinate data is (x c ,y c ).

[0094] S270. Based on the coordinate data of multiple lane line target points, the target lane line corresponding to the output data is formed.

[0095] For example, based on the coordinate data of multiple lane line target points, any two adjacent lane line target points of the first coordinate data are connected by a curve to obtain a curve, and this curve is determined as the target lane line corresponding to the output data.

[0096] For example, suppose the preset univariate polynomial function is a cubic polynomial function y = a³ × x 3 +a2×x 2 In the output data of the deep polynomial regression model, +a1×x+a0, the polynomial parameters include a3=0, a2=0.5, a1=0.2, and a0=3. The first coordinate data of the lane line starting point is 0, the first coordinate data of the lane line starting point is 300, and the preset coordinate interval is 2. Then 151 lane line target points can be determined. Based on the coordinate data of these 151 lane line target points, the curve y=0.5×x 2 The plotting of +0.2×x+3,x∈[0,300] yields the target lane line corresponding to the output data.

[0097] This embodiment samples multiple first-sampled coordinate data within the range of the first coordinate data corresponding to the first coordinate data of the lane line start point and the first coordinate data of the lane line end point. Then, it forms the coordinate data of a lane line target point by inputting each first-sampled coordinate data and the second coordinate data obtained by inputting the first-sampled coordinate data into the target polynomial function. This allows for the generation of coordinate data of multiple lane line target points for forming the target lane line through simple mathematical calculations, thereby improving the lane line calculation speed and saving computational resources.

[0098] Example 3

[0099] Figure 5 is a structural schematic diagram of a lane line detection device provided in Embodiment 3 of this application. This lane line detection device is configured in a vehicle equipped with an onboard radar. As shown in Figure 5, the device includes:

[0100] The 3D radar point cloud data acquisition module 310 is configured to acquire 3D radar point cloud data of the vehicle's driving direction environment.

[0101] The grid feature data determination module 320 is configured to perform grid division on the three-dimensional radar point cloud data to obtain grid feature data.

[0102] The output data determination module 330 is configured to input grid feature data into a deep multinomial regression model to obtain at least one set of output data; each set of output data includes polynomial parameters in a preset univariate multinomial function and the first coordinate data of the lane line target point; the first coordinate data is the coordinate data of the lane line target point in the driving direction;

[0103] The target lane line determination module 340 is configured to, for each set of output data, obtain a target polynomial function based on the polynomial parameters in the output data and a preset univariate multinomial polynomial function; and obtain the target lane line corresponding to each set of output data based on the target polynomial function and the first coordinate data of the lane line target point in the output data.

[0104] This embodiment inputs raster feature data into a deep polynomial regression model to obtain at least one set of output data. Based on the polynomial parameters in the output data and a preset univariate polynomial function, a target polynomial function is obtained. Based on the target polynomial function and the first coordinate data of the lane line target points in the output data, the target lane line corresponding to each set of output data is obtained. It eliminates the need to filter the points corresponding to the lane line based on confidence and perform curve fitting on the points corresponding to the lane line to obtain the target lane line. It can directly obtain the target lane line based on the output data and the preset univariate polynomial function, solving the problem of low lane line detection efficiency, improving lane line calculation speed, and saving computing resources.

[0105] Based on the above embodiments, optionally, the lane line target point includes a lane line start point and a lane line end point; the target polynomial function is used to characterize the relationship between the second coordinate data of the lane line target point and the first coordinate data of the lane line target point; the second coordinate data is the coordinate data of the lane line target point in the direction perpendicular to the driving direction; correspondingly, the target lane line determination module 340 is configured to: determine the range of first coordinate data corresponding to the output data based on the first coordinate data of the lane line start point and the first coordinate data of the lane line end point; sample within the range of first coordinate data to obtain multiple first sampled coordinate data; input each first sampled coordinate data into the target polynomial function to obtain the second coordinate data corresponding to each first sampled coordinate data; each first sampled coordinate data and the second coordinate data corresponding to the first sampled coordinate data form the coordinate data of a lane line target point; and form the target lane line corresponding to the output data based on the coordinate data of multiple lane line target points.

[0106] Based on the above embodiments, optionally, the target lane line determination module 340 is configured to sample within the first coordinate data range in the following manner to obtain multiple first sampled coordinate data: sampling the first coordinate data range corresponding to the output data at intervals based on a preset coordinate interval to obtain multiple first sampled coordinate data.

[0107] Based on the above embodiments, optionally, the three-dimensional radar point cloud data includes X-axis coordinate information, Y-axis coordinate information, Z-axis coordinate information, and intensity information corresponding to multiple target points respectively; correspondingly, the grid feature data determination module 320 is configured to: determine the cell grid to which each target point belongs based on the X-axis coordinate information and Y-axis coordinate information corresponding to each target point; determine the cell grid feature data corresponding to the cell grid based on the Z-axis coordinate information and intensity information corresponding to at least one target point contained in the cell grid, as well as the grid X-axis coordinate information and grid Y-axis coordinate information corresponding to the cell grid; and determine the grid feature data based on the cell grid feature data corresponding to multiple cell grids respectively.

[0108] Based on the above embodiments, optionally, the grid feature data determination module 320 is configured to determine the unit grid feature data corresponding to the unit grid by means of the following methods: determining the maximum Z-axis coordinate information, minimum Z-axis coordinate information, and average Z-axis coordinate information of the at least one target point contained in the unit grid, and the grid X-axis coordinate information and grid Y-axis coordinate information of the unit grid respectively; determining the maximum intensity information, minimum intensity information, and average intensity information of the intensity information of the at least one target point contained in the unit grid; determining the grid X-axis coordinate information of the unit grid based on the X-axis coordinate range of the unit grid, and determining the grid Y-axis coordinate information of the unit grid based on the Y-axis coordinate range of the unit grid; and determining the maximum Z-axis coordinate information, minimum Z-axis coordinate information, average Z-axis coordinate information, maximum intensity information, minimum intensity information, average intensity information, grid X-axis coordinate information, and grid Y-axis coordinate information as the unit grid feature data corresponding to the unit grid.

[0109] Based on the above embodiments, optionally, the preset univariate multinomial function is a univariate cubic polynomial function; the deep polynomial regression model includes a backbone network, a feature straightening layer, and a fully connected layer; wherein, the backbone network is used to extract lane line features from the raster feature data; the feature straightening layer is used to convert the lane line features into one-dimensional lane line features; and the fully connected layer is used to predict at least one set of output data based on the one-dimensional lane line features.

[0110] Based on the above embodiments, optionally, the lane line detection device further includes a three-dimensional radar point cloud data preprocessing module; wherein, the three-dimensional radar point cloud data preprocessing module is configured to preprocess the three-dimensional radar point cloud data before performing grid division on the three-dimensional radar point cloud data; wherein, the preprocessing includes removing invalid points; and converting the coordinate system of the preprocessed three-dimensional radar point cloud data into the vehicle body coordinate system.

[0111] The lane line detection device provided in this application embodiment can execute the lane line detection method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the execution method.

[0112] Example 4

[0113] Figure 6 is a schematic diagram of the structure of an electronic device implementing the lane detection method of this application embodiment. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples.

[0114] As shown in Figure 6, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0115] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0116] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as lane detection methods.

[0117] In some embodiments, the lane detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the lane detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the lane detection method by any other suitable means (e.g., by means of firmware).

[0118] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] Computer programs used to implement the lane detection method of this application can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0120] Example 5

[0121] Embodiment 5 of this application also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a lane line detection method, the method comprising:

[0122] Acquire 3D radar point cloud data of the vehicle's driving direction environment; divide the 3D radar point cloud data into grids to obtain grid feature data; input the grid feature data into a deep polynomial regression model to obtain at least one set of output data; each set of output data includes polynomial parameters in a preset univariate polynomial function and the first coordinate data of the lane line target point; the first coordinate data is the coordinate data of the lane line target point in the driving direction; for each set of output data, obtain a target polynomial function based on the polynomial parameters in the output data and the preset univariate polynomial function; obtain the target lane line corresponding to each set of output data based on the target polynomial function and the first coordinate data of the lane line target point in the output data.

[0123] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. Examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc-read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD), or monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0125] The systems and technologies described herein can be implemented in computing systems that include a backend (e.g., as a data server), a computing system that includes middleware (e.g., an application server), a computing system that includes a frontend (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such a backend, middleware, or frontend. The systems can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the Internet.

[0126] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0127] It should be understood that the various processes shown above can be used to reorder, add, or delete steps. For example, the multiple steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved.

Claims

1. A lane line detection method, applied to a vehicle equipped with onboard radar, the method comprising: Acquire three-dimensional radar point cloud data of the vehicle's driving direction environment; The three-dimensional radar point cloud data is divided into grids to obtain grid feature data; The raster feature data is input into a deep multinomial regression model to obtain at least one set of output data; each set of output data includes the polynomial parameters in a preset univariate multinomial function and the first coordinate data of the lane line target point. The first coordinate data is the coordinate data of the lane line target point in the driving direction; For each set of output data, a target polynomial function is obtained based on the polynomial parameters in the output data and a preset univariate polynomial function. The target lane line corresponding to each set of output data is obtained based on the objective polynomial function and the first coordinate data of the lane line target point in the output data.

2. The method according to claim 1, wherein, The lane line target points include the lane line start point and the lane line end point; The objective polynomial function is used to characterize the relationship between the second coordinate data of the lane line target point and the first coordinate data of the lane line target point; The second coordinate data is the coordinate data of the lane line target point in the direction perpendicular to the driving direction; The process of obtaining the target lane line corresponding to each set of output data based on the target polynomial function and the first coordinate data of the lane line target point in the output data includes: Based on the first coordinate data of the starting point of the lane line and the first coordinate data of the ending point of the lane line, the range of the first coordinate data corresponding to the output data is determined. Sampling is performed within the first coordinate data range to obtain multiple first sampled coordinate data. Each first sampled coordinate data is input into the target polynomial function to obtain the second coordinate data corresponding to each first sampled coordinate data. Each first sampled coordinate data and the second coordinate data corresponding to the first sampled coordinate data form the coordinate data of a lane line target point. The target lane line corresponding to the output data is formed based on the coordinate data of multiple lane line target points.

3. The method according to claim 2, wherein, The sampling within the first coordinate data range yields multiple first sampled coordinate data, including: Based on a preset coordinate interval, the range of first coordinate data corresponding to the output data is sampled at intervals to obtain multiple first sampled coordinate data.

4. The method according to claim 1, wherein, The three-dimensional radar point cloud data includes X-axis coordinate information, Y-axis coordinate information, Z-axis coordinate information, and intensity information corresponding to multiple target points; The step of dividing the three-dimensional radar point cloud data into grids to obtain grid feature data includes: Based on the X-axis coordinate information and Y-axis coordinate information corresponding to each target point, the cell grid to which each target point belongs is determined; Based on the Z-axis coordinate information and intensity information corresponding to at least one of the target points contained in the unit grid, and the grid X-axis coordinate information and grid Y-axis coordinate information corresponding to the unit grid, the unit grid feature data corresponding to the unit grid is determined. The grid feature data is determined based on the grid feature data corresponding to each of the multiple grid cells.

5. The method according to claim 4, wherein, The step of determining the cell feature data corresponding to the cell grid based on the Z-axis coordinate information and intensity information corresponding to at least one of the target points contained in the cell grid, and the X-axis coordinate information and Y-axis coordinate information corresponding to the cell grid, includes: Determine the maximum, minimum, and average Z-axis coordinate information of the Z-axis coordinate information corresponding to at least one of the target points contained in the cell grid; Determine the maximum intensity information, minimum intensity information, and average intensity information of the intensity information corresponding to at least one of the target points contained in the cell grid; Based on the X-axis coordinate range corresponding to the unit grid, determine the grid X-axis coordinate information corresponding to the unit grid; and based on the Y-axis coordinate range corresponding to the unit grid, determine the grid Y-axis coordinate information corresponding to the unit grid. The maximum Z-axis coordinate information, the minimum Z-axis coordinate information, the average Z-axis coordinate information, the maximum intensity information, the minimum intensity information, the average intensity information, the grid X-axis coordinate information, and the grid Y-axis coordinate information are determined as the unit grid feature data corresponding to the unit grid.

6. The method according to claim 1, wherein, The preset univariate multiple polynomial function is a univariate cubic polynomial function. The deep multinomial regression model includes a backbone network, a feature straightening layer, and a fully connected layer; wherein, the backbone network is used to extract lane line features from the raster feature data; the feature straightening layer is used to convert the lane line features into one-dimensional lane line features; and the fully connected layer is used to predict at least one set of output data based on the one-dimensional lane line features.

7. The method according to claim 1, wherein before performing grid division on the three-dimensional radar point cloud data, the method further comprises: The three-dimensional radar point cloud data is preprocessed; wherein, the preprocessing includes removing invalid points; The coordinate system of the preprocessed 3D radar point cloud data is converted into the vehicle body coordinate system.

8. A lane detection device, configured in a vehicle equipped with onboard radar, comprising: The three-dimensional radar point cloud data acquisition module is configured to acquire three-dimensional radar point cloud data of the driving direction environment of the vehicle. The grid feature data determination module is configured to perform grid division on the three-dimensional radar point cloud data to obtain grid feature data. The output data determination module is configured to input the raster feature data into a deep multinomial regression model to obtain at least one set of output data; each set of output data includes the polynomial parameters in a preset univariate multinomial function and the first coordinate data of the lane line target point. The first coordinate data is the coordinate data of the lane line target point in the driving direction; The target lane line determination module is configured to obtain a target polynomial function for each set of output data based on the polynomial parameters in the output data and a preset univariate polynomial function. The target lane line corresponding to each set of output data is obtained based on the objective polynomial function and the first coordinate data of the lane line target point in the output data.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the lane detection method according to any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the lane detection method of any one of claims 1-7.

Images