A lane line fitting method, device, equipment and storage medium

Abstract

A lane line fitting method, device, equipment and storage medium are disclosed. The lane line fitting method comprises: obtaining a lane line type point set in a vehicle body coordinate system based on vehicle pose information; determining the second derivative of each lane line type point according to the arrangement order of the lane line type points in the lane line type point set, and determining a segmentation point in the lane line type point set based on the second derivative; fitting a curve between adjacent segmentation points as an initial lane line; and performing smoothing processing on the initial lane line to obtain a final lane line. The technical scheme of the embodiment of the present application improves the accuracy and smoothness of the fitted lane line.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a lane line fitting method, apparatus, device, and storage medium. Background Technology

[0002] High-precision vector maps are a standard feature in advanced driver assistance systems, providing strong support for vehicle planning and control, positioning and navigation, and environmental perception.

[0003] Lane lines are an important element in high-precision vector maps. When using high-precision vector maps, the accuracy, continuity, and smoothness of lane lines are all very important.

[0004] A common method for fitting lane lines is to use cubic curves. However, in complex scenarios, it may be impossible to fit the entire lane line with a single cubic curve. Ensuring the accuracy and smoothness of the fitted lane lines has become a focus for autonomous driving professionals. Summary of the Invention

[0005] This invention provides a lane line fitting method, apparatus, device, and storage medium to ensure the accuracy and smoothness of the fitted lane lines.

[0006] According to one aspect of the present invention, a lane line fitting method is provided, comprising:

[0007] Based on vehicle pose information, obtain the set of lane line points in the vehicle coordinate system;

[0008] Based on the arrangement order of the lane alignment points in the set of lane alignment points, determine the second derivative of each lane alignment point, and based on the second derivative, determine the segmentation points in the set of lane alignment points;

[0009] Fit the curve between adjacent segment points as the initial lane line;

[0010] The initial lane lines are smoothed to obtain the final lane lines.

[0011] According to another aspect of the present invention, a lane line fitting device is provided, comprising:

[0012] The lane line shape point acquisition module is used to acquire the lane line shape point set in the vehicle coordinate system based on the vehicle pose information.

[0013] The segmentation point determination module is used to determine the second derivative of each lane alignment point according to the arrangement order of lane alignment points in the lane alignment point set, and to determine the segmentation point in the lane alignment point set based on the second derivative.

[0014] The initial lane line fitting module is used to fit the curve between adjacent segment points as the initial lane line.

[0015] The final lane line determination module is used to smooth the initial lane line to obtain the final lane line.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the lane line fitting method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the lane line fitting method according to any embodiment of the present invention.

[0021] The technical solution of this invention determines segmentation points in the set of lane alignment points based on the second derivative of the lane alignment points, and performs curve fitting between adjacent segmentation points. This achieves segmented fitting of the lane line based on the slope change rate of the lane line, which can improve the accuracy of lane line fitting. Furthermore, smoothing the initial lane line after fitting can make the lane line smooth and continuous, and more closely match the actual environment.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a lane line fitting method provided in Embodiment 1 of the present invention;

[0025] Figure 2a This is a flowchart of a lane line fitting method provided in Embodiment 2 of the present invention;

[0026] Figure 2b This is a schematic diagram showing that the lane alignment point set includes error points according to Embodiment 2 of the present invention;

[0027] Figure 2c This is a schematic diagram illustrating the process of removing erroneous points from the lane alignment point set according to Embodiment 2 of the present invention;

[0028] Figure 2d This is a schematic diagram of lane line segmentation according to Embodiment 2 of the present invention;

[0029] Figure 3a This is a flowchart of a lane line fitting method provided in Embodiment 3 of the present invention;

[0030] Figure 3b This is a flowchart of lane line fitting according to Embodiment 3 of the present invention;

[0031] Figure 3c This is a schematic diagram of coordinate transformation according to Embodiment 3 of the present invention;

[0032] Figure 4 This is a schematic diagram of the structure of a lane line fitting device according to Embodiment 4 of the present invention;

[0033] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the lane line fitting method of Embodiment 5 of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1 The flowchart illustrates a lane line fitting method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where lane lines are segmented and fitted. The method can be executed by a lane line fitting device, which can be implemented in hardware and / or software and can be configured in various general-purpose computing devices. Figure 1 As shown, the method includes:

[0038] S110. Based on vehicle pose information, obtain the set of lane line points in the vehicle coordinate system.

[0039] Vehicle pose information is used to represent the vehicle's position and attitude. Vehicle pose information includes the vehicle's position information and attitude information. For example, vehicle pose information can be obtained through a positioning device within the vehicle, such as the longitude, latitude, and heading angle of the vehicle's location.

[0040] In this embodiment of the invention, based on the vehicle's pose information, multiple lane alignment points of the vehicle's lane are extracted from the map database via an Electronic Horizon Provider (EHP). Specifically, within the vehicle's lane, the EHP acquires multiple lane alignment points within a set distance from the vehicle's current location along the vehicle's direction of travel, forming a set of lane alignment points.

[0041] The lane alignment points extracted by EHP from the map database are all in the geographic coordinate system. To facilitate subsequent data processing, the extracted lane alignment points need to be mapped from the geographic coordinate system to the vehicle coordinate system. Specifically, the lane alignment points can first be mapped from the geographic coordinate system to the Gaussian coordinate system, and then mapped from the Gaussian coordinate system to the vehicle coordinate system.

[0042] In a specific example, based on the vehicle's current position and attitude information, the EHP (Electronic Power Hierarchy Process) retrieves multiple lane alignment points within a certain distance (e.g., 1000 meters) ahead of the vehicle in the map database. This constitutes a lane alignment point set. Furthermore, these lane alignment points from the geographic coordinate system are mapped to the vehicle's coordinate system, resulting in a lane alignment point set in the vehicle's coordinate system, which facilitates subsequent data fitting and processing.

[0043] It is worth noting that the original lane alignment point data in the map database may contain outliers. To ensure the accuracy of lane alignment fitting, after obtaining the set of lane alignment points in the vehicle coordinate system, outliers can be removed. For example, the Random Sample Consensus (RANSAC) algorithm can be used to remove outliers from the set of lane alignment points.

[0044] S120. Based on the arrangement order of lane alignment points in the lane alignment point set, determine the second derivative of each lane alignment point, and based on the second derivative, determine the segmentation points in the lane alignment point set.

[0045] In cases of complex lane markings, the entire lane line may not be describable by a single cubic curve. To improve the accuracy of lane line fitting, the lane line is fitted in segments.

[0046] In this embodiment of the invention, the second derivative of each lane alignment point is determined according to the arrangement order of the lane alignment points in the lane alignment point set. Then, based on the second derivative, several segmentation points are determined in the lane alignment point set to divide the lane line into multiple segments, and a lane line segment is fitted between every two adjacent segmentation points. Specifically, lane alignment points are sequentially extracted as the current lane alignment point, and the two adjacent lane alignment points before and after the current lane alignment point are obtained according to the arrangement order of the lane alignment points in the lane alignment point set. Further, the slope of two adjacent lane alignment points among the three lane alignment points is calculated, and then the rate of change of the slope is calculated as the second derivative of the current lane alignment point.

[0047] Furthermore, the second derivative of the current lane alignment point is compared with a set threshold. If the second derivative is greater than the threshold, it indicates a significant change in curvature of the lane line at the current lane alignment point. To ensure the accuracy of lane line fitting at this location, the current lane alignment point is used as a segmentation point. Based on multiple segmentation points, the lane line is divided into multiple segments. Segmentation based on the second derivative of the lane alignment point allows for fewer segments on straight road sections and more segments on curved road sections, ensuring curve fitting efficiency while improving fitting accuracy for curved road sections.

[0048] In a specific example, the set of lane alignment points includes three lane alignment points A, B, and C in sequence. When point B is the current lane alignment point, the slopes of segments AB and BC are calculated separately, and then the rate of change between the two slopes is calculated as the second derivative of point B. The second derivative is then compared with a set threshold. If it is greater than the threshold, point B is designated as the segmentation point.

[0049] S130. Fit the curve between adjacent segment points as the initial lane line.

[0050] In this embodiment of the invention, based on the lane alignment points between adjacent segment points and the curve between two adjacent segment points, an initial lane line is fitted. Specifically, curve fitting can be performed using matrix decomposition to obtain the initial lane line; however, the curve fitted by matrix decomposition may have significant errors. Therefore, curve fitting can also be performed using the least squares method to obtain the initial lane line. Directly using the least squares method for curve fitting is computationally intensive and prone to getting trapped in local optima. Therefore, it is also possible to first perform curve fitting using matrix decomposition to obtain preliminary curve coefficients, and then use the least squares method to adjust the preliminary curve coefficients of the fitted curve to obtain the initial lane line, which improves computational efficiency while avoiding getting trapped in local optima.

[0051] S140. Smooth the initial lane lines to obtain the final lane lines.

[0052] In S130, each lane line segment has been fitted separately. However, during the segmented fitting process, only the accuracy of the lane lines within each segment is considered, without taking into account the continuity and smoothness of the entire lane. If there are non-smooth transitions between segments, it will affect the user experience.

[0053] In this embodiment of the invention, after fitting each segment of the lane line to obtain multiple initial lane lines, the initial lane lines are further optimized based on the idea of ​​global optimization, so that the transition between segments is smooth.

[0054] Specifically, the first and second derivatives of each initial lane line segment at the segmentation point can be calculated. For the difference in the first and second derivatives of the same segmentation point in two different initial lane lines, the least squares method is used to optimize the curve coefficient of the initial lane line, thus obtaining the optimized final lane line.

[0055] Alternatively, the second derivative of each lane line with the current segment point as its endpoint can be calculated, and the average of the second derivatives can be obtained. If the average second derivative is less than or equal to K1, the slope change rate at that point is considered small, and the probability of fitting error is low. In this case, to reduce computational load, the first derivative of the current segment point in the two connected initial lane lines is used as the optimization index parameter. If the average second derivative is greater than K1 and less than K2 (K2>K1), the slope change rate at that point is considered large, and the probability of fitting error is high. In this case, the first and second derivatives of the current segment point in the two connected initial lane lines are used as the optimization index parameters. If the average second derivative is equal to or greater than K2, the slope change rate at that point is considered large, and the probability of an uneven transition between the initial lane lines at both ends is high. In this case, the second derivative of the two connected initial lane lines is used as the optimization index parameter. Finally, based on the optimization index parameters for each segment point, the least squares method is used for optimization to obtain the optimized final lane line.

[0056] The technical solution of this invention determines segmentation points in the set of lane alignment points based on the second derivative of the lane alignment points, and performs curve fitting between adjacent segmentation points. This achieves segmented fitting of the lane line based on the slope change rate of the lane line, which can improve the accuracy of lane line fitting. Furthermore, smoothing the initial lane line after fitting can make the lane line smooth and continuous, and more closely match the actual environment.

[0057] Example 2

[0058] Figure 2a This is a flowchart of a lane line fitting method provided in Embodiment 2 of the present invention. This embodiment further refines the above embodiment, providing specific steps for determining segment points in the lane line shape point set based on the second derivative, and specific steps for fitting the curve between adjacent segment points as the initial lane line. Figure 2a As shown, the method includes:

[0059] S210. Based on vehicle pose information, obtain the set of lane line points in the vehicle coordinate system.

[0060] S220. The Random Sample Consensus (RANSAC) algorithm is used to remove erroneous points from the lane alignment point set.

[0061] In embodiments of the present invention, such as Figure 2bAs shown, the lane alignment point set obtained in S210 may contain erroneous points that affect the accuracy of lane alignment fitting. To ensure the accuracy of lane alignment fitting, the Random Sample Consensus (RANSAC) algorithm is used before lane alignment fitting to remove erroneous points from the lane alignment point combination, resulting in the following... Figure 2c The set of lane alignment points shown.

[0062] S230. Determine the second derivative of each lane alignment point according to the arrangement order of the lane alignment points in the lane alignment point set.

[0063] S240. Sequentially extract lane alignment points from the set of lane alignment points as the current lane alignment points, and compare the second derivative of the current lane alignment point with a set threshold.

[0064] In this embodiment of the invention, lane alignment points are sequentially extracted from the set of lane alignment points as current lane alignment points. Then, the second derivative of the current lane alignment point is compared with a set threshold to determine the slope change rate of the lane line at the current lane alignment point. The segmentation method of the lane line is then adaptively adjusted according to the slope change rate.

[0065] S250. If the second derivative of the current lane alignment point is greater than a set threshold, determine the current lane alignment point as a segment point.

[0066] In this embodiment of the invention, if the second derivative of the current lane alignment point is greater than a set threshold, it indicates that the rate of change of the slope of the lane line at the current lane alignment point is large, which is prone to fitting errors. To reduce fitting errors, the current lane alignment point is used as a segmentation point to segment the lane line. Segmenting at lane alignment points where the second derivative is greater than the threshold can achieve the effect of fewer segments for straight road segments and more segments for curved road segments, thereby improving the fitting efficiency of straight road segments and improving the fitting accuracy of curved road segments.

[0067] S260. Fit the curves between adjacent segment points using matrix decomposition to obtain the initial curve.

[0068] In this embodiment of the invention, after segmenting the lane lines, the curves between adjacent lane lines are fitted by matrix decomposition based on adjacent segment points and other lane line points between adjacent segment points to obtain an initial curve.

[0069] S270. The initial curve is adjusted using the least squares method to obtain the initial lane line.

[0070] In this embodiment of the invention, based on the initial curve obtained in S260, the least squares method is further used to adjust the initial curve to obtain the initial lane lines in order to improve the fitting accuracy. Specifically, based on the least squares method, the curve coefficients in the initial curve are optimized to minimize the sum of squares of the errors between the actual and calculated values ​​of the lane alignment points in the lane alignment point set. Through the secondary adjustment using the least squares method, the fitting accuracy of each segment of the initial lane line is improved.

[0071] like Figure 2d As shown, in the set of lane line points, if the second derivatives of points M and N are greater than a set threshold, they are used as segmentation points. The lane line is then divided into segments PM, MN, and NQ, which are fitted separately to obtain the initial lane line.

[0072] S280. Smooth the initial lane lines to obtain the final lane lines.

[0073] The technical solution of this invention, after obtaining the set of lane alignment points in the vehicle coordinate system, uses the RANSAC algorithm to remove erroneous points from the set of lane alignment points, which can avoid the influence of erroneous points on lane alignment fitting. Lane alignment points with second derivatives greater than a set threshold are used as segmentation points, realizing fewer segments for straight road segments and more segments for curved road segments. This reduces the amount of computation and improves the fitting accuracy of curved road segments. Furthermore, after fitting each lane line segment using matrix decomposition, the least squares method is used for adjustment. This reduces the amount of fitting computation and avoids the fitted curve coefficients from getting trapped in local optima, further improving the accuracy of the fitted lane lines.

[0074] Example 3

[0075] Figure 3a This is a flowchart of a lane line fitting method provided in Embodiment 3 of the present invention. This embodiment further refines the above embodiments, providing specific steps for obtaining a set of lane line points in the vehicle coordinate system based on vehicle pose information, and specific steps for smoothing the initial lane line to obtain the final lane line. Figure 3a As shown, the method includes:

[0076] S310. Within the lane where the vehicle is located, starting from the vehicle's current position, obtain the lane alignment points in the geographic coordinate system within a set distance along the vehicle's direction of travel.

[0077] In this embodiment of the invention, the lane fitting process is as follows: Figure 3bAs shown, within the vehicle's lane, EHP retrieves lane alignment points in the map database within a set distance along the vehicle's direction of travel, using the vehicle's current location as the starting point. For example, within the vehicle's lane, multiple lane alignment points within 1000 meters ahead of the vehicle are retrieved along the vehicle's direction of travel to fit the lane line.

[0078] S320. Determine the target map grid to which the lane alignment points in the geographic coordinate system belong, and map the lane alignment points in the geographic coordinate system to the Gaussian coordinate system based on the coordinate transformation values ​​associated with the target map grid.

[0079] The coordinate transformation values ​​are obtained by dividing the Earth's latitude into multiple map grids and calculating them based on the latitude and longitude coordinates of the map grids.

[0080] In this embodiment of the invention, after extracting lane alignment points in the geographic coordinate system, it is necessary to map these points to the vehicle coordinate system for easier subsequent calculations. However, due to the high frequency of map data transmission and calculation, and the large number of lane alignment points, it is necessary to accelerate the coordinate transformation rate to improve computational efficiency. Since the coordinate transformation matrix (the matrix for converting latitude and longitude coordinates to Gaussian coordinates) is only related to latitude, the Earth is pre-divided into grids to speed up the transformation. For example, for the Northern Hemisphere, latitude 0°N to 90°N is divided into 256 equal parts, and the coordinate transformation value (i.e., the coordinate value of the point in the Gaussian coordinate system) is pre-calculated for each part.

[0081] Specifically, such as Figure 3c As shown, when calculating the coordinate transformation value of point X between points A and B, the coordinate value of point X is calculated based on the ratio of the distances from point X to points A and B, i.e., (XA) / a = (BA) / (a+b), where A represents the coordinate value of point A in the Gaussian coordinate system, B represents the coordinate value of point B in the Gaussian coordinate system, a represents the distance from point X to point A, and b represents the distance from point X to point B. This allows us to obtain the coordinate value of point X in the Gaussian coordinate system. Using this dictionary lookup method can improve the efficiency of coordinate transformation, thereby improving the efficiency of lane line fitting.

[0082] S330. Based on the vehicle pose information, the lane line points in the Gaussian coordinate system are mapped to the vehicle body coordinate system to obtain the set of lane line points in the vehicle body coordinate system.

[0083] In this embodiment of the invention, a vehicle coordinate system is established based on vehicle pose information, and the coordinate values ​​in the Gaussian coordinate system are mapped to the vehicle coordinate system to obtain a set of lane line points in the vehicle coordinate system. This greatly improves the calculation efficiency for converting a large number of lane line points to the vehicle coordinate system.

[0084] S340. The Random Sample Consensus (RANSAC) algorithm is used to remove erroneous points from the lane alignment point set.

[0085] S350. Based on the arrangement order of lane alignment points in the lane alignment point set, determine the second derivative of each lane alignment point, extract lane alignment points from the lane alignment point set in sequence as the current lane alignment point, and compare the second derivative of the current lane alignment point with a set threshold.

[0086] S360. If the second derivative of the lane alignment point is greater than a set threshold, the current lane alignment point is determined as a segmentation point.

[0087] S370. Fit the curves between adjacent segment points using matrix decomposition to obtain the initial curve.

[0088] S380. The initial curve is adjusted using the least squares method to obtain the initial lane line.

[0089] S390. Traverse the segment points in the set of lane alignment points, and calculate the first and second derivatives of the current segment point in the two initial lane lines connected by the current segment point.

[0090] Because the above process involves segmenting and fitting lane lines, it only considers the accuracy and smoothness of lane lines within a single segment, without considering the smooth connections between segments. Therefore, the resulting initial lane lines may have uneven transitions between segments, affecting the user experience. In this embodiment of the invention, the initial lane lines are further optimized globally to ensure smoothness between segments, making the entire lane line continuous and smooth.

[0091] Specifically, first, the segment points in the lane alignment point set are traversed, and one segment point is extracted as the current segment point. The two initial lane lines with the endpoints of the current segment point are determined, and the first and second derivatives of the two initial lane lines at the current segment point are calculated. These are used as optimization parameters to adjust the initial lane lines.

[0092] S391. Based on the first and second derivatives of each segment point in the two connected initial lane lines, the initial lane lines are smoothed using the least squares method to obtain the final lane lines.

[0093] In this embodiment of the invention, based on the first and second derivatives of each segment point in the two connected initial lane lines, the least squares method is used to smooth the initial lane lines to obtain the final lane lines. Specifically, the difference between the first and second derivatives of each segment point in the two connected lane lines can be calculated, thereby minimizing the sum of the squares of the differences between the first and second derivatives at all segment points to obtain the final lane lines.

[0094] Specifically, the mean of the second derivatives of the segment points in the two connected initial lane lines can be calculated to determine the magnitude of the slope change rate at that segment point. If the mean of the second derivatives of the current segment point is greater than a threshold, it indicates a large slope change rate at that point, requiring optimization of smoothness. Therefore, the difference between the second derivatives of the current segment point in the two connected initial lane lines is used as the error value. If the mean of the second derivatives of the current segment point is less than a threshold, it indicates a small slope change rate at that point, requiring a focus on fitting accuracy rather than smoothness. Therefore, the difference between the first derivatives of the current segment point in the two connected initial lane lines is used as the error value. Finally, the least squares method is used to minimize the sum of the squares of the error values ​​at each segment point, resulting in the final lane line.

[0095] Optionally, based on the first and second derivatives of each segment point in the two connected initial lane lines, the initial lane lines are smoothed using the least squares method to obtain the final lane lines, including:

[0096] Traverse the segment points in the set of lane alignment points, calculate the mean of the second derivatives of the current segment point in the two initial lane lines it connects, and use it as the mean second derivative of the current segment point;

[0097] If the average second derivative of the current segment point is less than or equal to the first value, the first derivative of the segment point in the two connected initial lane lines is used as the optimization index parameter of the current segment point.

[0098] If the average second derivative of the current segment point is greater than the first value and less than the second value, the first and second derivatives of the current segment point in the two initial lane lines connected to it are used as the optimization index parameters of the current segment point; the first value is less than the second value.

[0099] If the average second derivative of the current segment point is equal to or greater than the second data value, the second derivative of the current segment point in the two initial lane lines connected to it will be used as the optimization index parameter of the current segment point.

[0100] Based on the optimized index parameters of each segment point, the initial lane line is smoothed using the least squares method to obtain the final lane line.

[0101] In this optional embodiment, a method is provided to smooth the initial lane lines using the least squares method based on the first and second derivatives of each segment point in the two connected initial lane lines to obtain the final lane line: First, traverse the segment points in the lane line shape point set, and sequentially take the extracted segment points as the current segment point. Calculate the average of the second derivatives of the current segment point in the two connected initial lane lines as the average second derivative of the current segment point. Further, if the average second derivative of the current segment point is less than or equal to a first value, it is considered that the slope change rate of the current segment point is small, and the possibility of unevenness between the two segments is small. Therefore, using the first derivative of the segment point in the two connected initial lane lines as the optimization index parameter of the current segment point can improve the computational efficiency in the optimization process. If the average second derivative of the current segment point is greater than the first value and less than the second value, it is considered that the slope change rate of the current segment point is large, and it is necessary to consider both the fitting accuracy and smoothness at this point. Therefore, the average second derivative of the current segment point is used as the optimization index parameter. The first and second derivatives of the previous segment point in the two connected initial lane lines are used as optimization index parameters for the current segment point to ensure the accuracy and smoothness of the optimized lane line. The first value is less than the second value. If the average second derivative of the current segment point is equal to or greater than the second value, the slope change rate of the current segment point is considered large, requiring close attention to its smoothness. Therefore, using the second derivative of the current segment point in the two connected initial lane lines as the optimization index parameter improves the smoothness at the current segment point. Finally, based on the optimization index parameters of each segment point, the initial lane line is smoothed using the least squares method to obtain the final lane line. Specifically, the difference in the optimization index parameters of each segment point in the two connected initial lane lines is calculated, and the sum of squares of these differences is taken. The least squares method is then used to minimize this sum of squares to obtain the final lane line.

[0102] In a specific example, the lane line includes segment points A, B, and C. The average second derivative of each segment point in the two connected initial lane lines is calculated. Specifically, the average second derivative of segment point A is less than a first value, the average second derivative of segment point B is greater than the first value but less than a second value, and the average second derivative of segment point C is greater than the second value. Further, the first derivative of segment point A in the two connected initial lane lines is calculated, and the difference between the first derivatives of the two segments is calculated; the first and second derivatives of segment point B in the two connected initial lane lines are calculated, and the difference between the first and second derivatives of the two segments is calculated; the second derivative of segment point C in the two connected initial lane lines is calculated, and the difference between the second derivatives of the two segments is calculated. Finally, the least squares method is used to minimize the sum of the squares of all the above differences, resulting in the optimized final lane line.

[0103] The above-mentioned method of determining the optimization index parameters to be considered based on the average second derivative for global optimization allows for flexible adjustment of the optimization parameters to be considered for the slope change rate at different segment points. This improves computational efficiency while ensuring the fitting accuracy and smoothness of the final lane line after optimization.

[0104] The technical solution of this invention improves the efficiency of coordinate transformation by mapping the coordinates of lane line points to a Gaussian coordinate system through a map grid obtained by pre-dividing the Earth into grids. In addition, based on the first and second derivatives of each segment point in the set of lane line points in the two initial lane lines it connects to, the initial lane lines are smoothed to obtain the final lane lines, making the connection between each segment of the lane lines smoother and more continuous, thus improving the user experience.

[0105] Example 4

[0106] Figure 4 This is a schematic diagram of a lane line fitting device provided in Embodiment 4 of the present invention. Figure 4 As shown, the device includes:

[0107] The lane line shape point acquisition module 410 is used to acquire the lane line shape point set in the vehicle body coordinate system based on the vehicle pose information.

[0108] The segmentation point determination module 420 is used to determine the second derivative of each lane alignment point according to the arrangement order of lane alignment points in the lane alignment point set, and to determine the segmentation point in the lane alignment point set based on the second derivative.

[0109] The initial lane line fitting module 430 is used to fit the curve between adjacent segment points as the initial lane line.

[0110] The final lane line determination module 440 is used to smooth the initial lane line to obtain the final lane line.

[0111] The technical solution of this invention determines segmentation points in the set of lane alignment points based on the second derivative of the lane alignment points, and performs curve fitting between adjacent segmentation points. This achieves segmented fitting of the lane line based on the slope change rate of the lane line, which can improve the accuracy of lane line fitting. Furthermore, smoothing the initial lane line after fitting can make the lane line smooth and continuous, and more closely match the actual environment.

[0112] Optional, the point set acquisition module 410 is specifically used for:

[0113] Within the lane where the vehicle is located, starting from the current position of the vehicle, obtain the lane line points in the geographic coordinate system within a set distance along the direction of the vehicle's travel;

[0114] Determine the target map grid to which the lane alignment points belong in the geographic coordinate system, and map the lane alignment points in the geographic coordinate system to the Gaussian coordinate system based on the coordinate transformation values ​​associated with the target map grid.

[0115] The coordinate transformation values ​​are obtained by dividing the Earth's latitude into multiple map grids and calculating them based on the latitude and longitude coordinates of the map grids.

[0116] Based on the vehicle pose information, the lane line points in the Gaussian coordinate system are mapped to the vehicle body coordinate system to obtain the set of lane line points in the vehicle body coordinate system.

[0117] Optional, the segmentation point determination module 420 is specifically used for:

[0118] The lane alignment points are extracted sequentially from the set of lane alignment points as the current lane alignment points, and the second derivative of the current lane alignment point is compared with a set threshold.

[0119] If the second derivative of the current lane alignment point is greater than a set threshold, the current lane alignment point is determined as a segmentation point.

[0120] Optional, the initial lane line fitting module 430 is specifically used for:

[0121] An initial curve is obtained by fitting the curve between adjacent segment points using matrix decomposition.

[0122] The initial curve is adjusted using the least squares method to obtain the initial lane line.

[0123] Optionally, the final lane line determination module 440 includes:

[0124] The derivative calculation unit is used to traverse the segment points in the lane alignment point set and calculate the first and second derivatives of the current segment point in the two initial lane lines connected by the current segment point.

[0125] The final lane line determination unit is used to smooth the initial lane line based on the first and second derivatives of each segment point in the two connected initial lane lines, using the least squares method, to obtain the final lane line.

[0126] Optional, the final lane line determination unit is specifically used for:

[0127] Traverse the segment points in the set of lane alignment points, calculate the mean of the second derivatives of the current segment point in the two initial lane lines it connects, and use it as the mean second derivative of the current segment point;

[0128] If the average second derivative of the current segment point is less than or equal to the first value, the first derivative of the segment point in the two connected initial lane lines is used as the optimization index parameter of the current segment point.

[0129] If the average second derivative of the current segment point is greater than the first value and less than the second value, the first and second derivatives of the current segment point in the two initial lane lines connected to it are used as the optimization index parameters of the current segment point; the first value is less than the second value.

[0130] If the average second derivative of the current segment point is equal to or greater than the second data value, the second derivative of the current segment point in the two initial lane lines connected to it will be used as the optimization index parameter of the current segment point.

[0131] Based on the optimized index parameters of each segment point, the initial lane line is smoothed using the least squares method to obtain the final lane line.

[0132] Optionally, the lane line fitting device also includes:

[0133] The error point removal module is used to remove error points in the lane alignment point set by employing the Random Sample Consensus (RANSAC) algorithm before determining the second derivative of each lane alignment point based on the arrangement order of the lane alignment points in the lane alignment point set.

[0134] The lane line fitting device provided in the embodiments of the present invention can execute the lane line fitting method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0135] Example 5

[0136] Figure 5 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0137] like Figure 5As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0138] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0139] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as lane line fitting methods.

[0140] In some embodiments, the lane fitting method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the lane fitting method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the lane fitting method by any other suitable means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0145] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0146] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0147] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0148] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A lane line fitting method, characterized by, The method comprises the following steps: acquiring a set of lane line points in a vehicle body coordinate system based on vehicle pose information; determining the second derivative of each lane line point according to the arrangement order of the lane line points in the set of lane line points, and determining a segmentation point in the set of lane line points based on the second derivative; fitting a curve between adjacent segmentation points as an initial lane line; traversing the segmentation points in the set of lane line points, and calculating the first derivative and the second derivative of the current segmentation point in the two initial lane lines connected by the current segmentation point; traversing the segmentation points in the set of lane line points, and calculating the average second derivative of the current segmentation point in the two initial lane lines connected by the current segmentation point as the average second derivative of the current segmentation point; in the case that the average second derivative of the current segmentation point is less than or equal to a first value, taking the first derivative of the current segmentation point in the two initial lane lines connected by the current segmentation point as the optimization index parameter of the current segmentation point; in the case that the average second derivative of the current segmentation point is greater than the first value and less than a second value, taking the first derivative and the second derivative of the current segmentation point in the two initial lane lines connected by the current segmentation point as the optimization index parameter of the current segmentation point; the first value is less than the second value; in the case that the average second derivative of the current segmentation point is equal to or greater than the second value, taking the second derivative of the current segmentation point in the two initial lane lines connected by the current segmentation point as the optimization index parameter of the current segmentation point; based on the optimization index parameter of each segmentation point, performing smoothing processing on the initial lane line by using the least square method to obtain a final lane line.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring a set of lane line points in a vehicle body coordinate system based on vehicle pose information; acquiring lane line points in a geographical coordinate system within a set distance along the direction of vehicle travel from the current position of the vehicle in the lane where the vehicle is located; determining the target map grid to which the lane line points in the geographical coordinate system belong, and mapping the lane line points in the geographical coordinate system to the Gauss coordinate system according to the coordinate conversion value associated with the target map grid; wherein the coordinate conversion value is obtained by dividing the latitude of the earth into a plurality of map grids and calculating based on the latitude and longitude coordinates of the map grid; 3. The method of claim 1, wherein, mapping the lane line points in the Gauss coordinate system to the vehicle body coordinate system based on the vehicle pose information to obtain a set of lane line points in the vehicle body coordinate system. The method comprises the following steps: extracting lane line points in the set of lane line points as the current lane line point in turn, and comparing the second derivative of the current lane line point with a set threshold value; 4. The method of claim 1, wherein, in the case that the second derivative of the current lane line point is greater than the set threshold value, determining the current lane line point as a segmentation point. The method comprises the following steps: fitting a curve between adjacent segmentation points by matrix decomposition to obtain an initial curve; 5. The method according to any of claims 1 to 4, characterized in that, adjusting the initial curve by using the least square method to obtain the initial lane line. Before determining the second derivative of each lane line point according to the arrangement order of the lane line points in the set of lane line points, the method further comprises the following steps: The random sample consensus (RANSAC) algorithm is used to remove the error points in the lane line point set.

6. A lane line fitting apparatus characterized by, The method comprises the following steps: The lane line point set in the vehicle body coordinate system is obtained based on the vehicle pose information. The second derivative of each lane line point is determined according to the arrangement order of the lane line points in the lane line point set, and the segment points are determined in the lane line point set based on the second derivative. The curve between adjacent segment points is fitted as the initial lane line. The final lane line determination module comprises: The first derivative and the second derivative of the current segment point are calculated in the two initial lane lines connected by the current segment point. The average second derivative of the current segment point is calculated by averaging the second derivatives of the current segment point in the two initial lane lines connected by the current segment point; the first derivative of the current segment point in the two initial lane lines connected by the current segment point is used as the optimization index parameter of the current segment point when the average second derivative of the current segment point is less than or equal to the first value; the first derivative and the second derivative of the current segment point in the two initial lane lines connected by the current segment point are used as the optimization index parameter of the current segment point when the average second derivative of the current segment point is greater than the first value and less than the second value; the second derivative of the current segment point in the two initial lane lines connected by the current segment point is used as the optimization index parameter of the current segment point when the average second derivative of the current segment point is equal to or greater than the second value; the initial lane line is smoothed by the least square method based on the optimization index parameter of each segment point, and the final lane line is obtained.

7. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the lane line fitting method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the lane line fitting method of any one of claims 1-5 when executed.

Images