Lane line fitting optimization method, device, computer equipment and storage medium

By using tracking and fusion technology of lane line point cloud in assisted driving system, combined with sampling strategies and optimization algorithms, fit lane lines in three-dimensional space, the problem of ignoring three-dimensional space in the existing technology is solved. Improve detection accuracy and robustness.

CN118552656BActive Publication Date: 2024-11-08SUZHOU MAGIC VISION INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411027914.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-08
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The prior art detects and fits the 2D lane line points in the picture in the assisted driving system, ignoring the actual layout in the three-dimensional space, resulting in a decrease in detection accuracy in complex three-dimensional traffic environments, and relies on mathematical algorithms to 2D lanes. Line fitting lacks optimization methods for three-dimensional spatial characteristics, which limits the improvement of detection performance.

Method used

By obtaining the lane line point clouds at each moment, tracking and fusion, determining the lane line point cloud set, determining the target path and control point based on the point cloud set, using sampling strategies and optimization algorithms to fit the lane line in three-dimensional space, and generating the target Lane line.

Benefits of technology

It improves the accuracy and robustness of lane line detection, can more accurately reflect the position and shape of lane line in three-dimensional space, enhances the accuracy and robustness of detection, and is suitable for complex and dynamically changing traffic environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118552656B_ABST
    Figure CN118552656B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data processing technology, and discloses a lane line fitting optimization method, device, computer equipment and storage medium. The method comprises: obtaining lane line point clouds at various moments, and tracking and fusing each lane line point cloud to determine a lane line point cloud set; determining a target path corresponding to the lane line point cloud based on the lane line point cloud set, and determining at least one control point based on the target path; determining sampling points corresponding to the control points based on the sampling strategy and the control points, and determining the target points of the lane lines of the adjacent lanes of the lane line based on the sampling points; optimizing the lane line point cloud based on the target points to obtain a first target control point, and generating a target lane line based on the first target control point. By determining the target path and control points, the present invention further utilizes three-dimensional information to guide the detection of lane lines, so that it is no longer solely dependent on the parallel constraints in the 2D image, but can more accurately fit the lane lines in three-dimensional space.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a lane line fitting optimization method, device, computer equipment and storage medium. Background Art

[0002] In the assisted driving system, the deep learning-based perception module is a key component, responsible for extracting useful information from complex environments, such as lane lines, to support subsequent positioning and decision-making functions.

[0003] At present, in order to improve the accuracy and robustness of lane line detection, manual annotation and automatic learning methods are used. Specifically, the scheme first uses limited manual annotation data to train an initial deep learning network model, and then uses the model to predict a large number of new unlabeled scenes. For the prediction results, they are corrected and improved through a series of post-processing optimization steps, and then these optimized data are re-incorporated into the training set, and the model performance is gradually improved through iterative training. Among them, in the post-processing optimization stage of lane line detection, it mostly relies on the predicted 2D lane line points in the picture, and fits them in combination with the geometric constraints of parallel lane lines. This method uses mathematical algorithms (such as Hough transform, least squares method or RANSAC, etc.) to fit the lane lines with straight lines or curves to improve the accuracy and continuity of lane line detection.

[0004] However, detection and fitting are performed based on 2D lane line points in the image, ignoring the actual layout in three-dimensional space, which results in reduced detection accuracy in complex three-dimensional traffic environments. In addition, the 2D lane line fitting relies on mathematical algorithms and lacks optimization methods for three-dimensional space characteristics, which limits further improvement of detection performance. Summary of the invention

[0005] In view of this, the present invention provides a lane line fitting optimization method, device, computer equipment and storage medium to solve the problem that detection and fitting are performed through 2D lane line points in an image, ignoring the actual layout in three-dimensional space, resulting in reduced detection accuracy in complex three-dimensional traffic environments, and relying on mathematical algorithms to fit 2D lane lines, lacking optimization methods for three-dimensional space characteristics, limiting the further improvement of detection performance.

[0006] In a first aspect, the present invention provides a lane line fitting optimization method, the method comprising: obtaining lane line point clouds at various moments, and tracking and fusing each lane line point cloud to determine a lane line point cloud set; determining a target path corresponding to the lane line point cloud based on the lane line point cloud set, and determining at least one control point based on the target path; determining sampling points corresponding to the control points based on a sampling strategy and control points, and determining target points of lane lines of adjacent lanes of the lane line based on the sampling points; optimizing the lane line point cloud based on the target point to obtain a first target control point, and generating a target lane line based on the first target control point.

[0007] The lane line fitting optimization method provided in this embodiment obtains the lane line point cloud at each moment and constructs a lane line point cloud set using tracking and fusion technology, which provides rich three-dimensional information for lane line detection, thereby more accurately reflecting the actual position and shape of the lane line in three-dimensional space, thereby improving the detection accuracy. In addition, by determining the target path and control points, the three-dimensional information is further used to guide the detection of the lane line, so that it is no longer solely dependent on the parallel constraints in the 2D image, but can more accurately fit the lane line in three-dimensional space. In addition, the process of optimizing the lane line point cloud through the target point, obtaining the first target control point and generating the target lane line fully utilizes the advantages of three-dimensional information, and directly adjusts and improves the lane line in three-dimensional space through the optimization algorithm, thereby improving the accuracy and robustness of the detection.

[0008] In an optional embodiment, each lane line point cloud is tracked and fused to determine a lane line point cloud set, including: obtaining a first lane line point cloud of a first lane line at a first moment and a second lane line point cloud of a second lane line at a second moment; wherein the first moment is used to represent an initial moment, and each first lane line point cloud constitutes an initial lane line point cloud set; based on the first lane line point cloud and the second lane line point cloud, determining a minimum distance between the first lane line and the second lane line; detecting whether the minimum distance is greater than a preset distance; if the minimum distance is less than the preset distance, fusing the first lane line point cloud and the second lane line point cloud to obtain a lane line point cloud set; if the minimum distance is greater than the preset distance, merging the second lane line point cloud into the initial lane line point cloud set to obtain a lane line point cloud set.

[0009] The lane line fitting optimization method provided in this embodiment can capture the dynamic changes of the lane line in real time by acquiring the lane line point clouds at the first moment and the second moment (and more subsequent moments), and then accurately determine the relative position relationship between the first lane line and the second lane line by calculating the minimum distance between the two, so that the lane line point cloud can be more accurately fused or updated in subsequent processing.

[0010] In an optional embodiment, based on a lane line point cloud set, a target path corresponding to the lane line point cloud is determined, and based on the target path, at least one control point is determined, including: constructing and processing the lane line point cloud set to obtain an undirected graph; downsampling the lane line point cloud set based on the first step length to determine the target lane line point cloud set; using the Dijkstra algorithm to determine the shortest path between each point in the target lane line point cloud set; comparing the shortest paths, and taking the longest path in the shortest paths as the target path; determining the initial control point based on the length of the target path; and determining the control point based on the initial control point and the point cloud of the target path using a third-order B-spline algorithm.

[0011] The lane line fitting optimization method provided in this embodiment can significantly reduce the amount of data and the computational complexity of subsequent processing by downsampling the lane line point cloud set. At the same time, key information points are retained to improve the accuracy of the processing results. In addition, by searching the shortest path between each point in the target lane line point cloud set through the Dijkstra algorithm, the optimal path connecting the starting point and the end point, that is, the target path, can be accurately found, thereby accurately reflecting the actual direction of the lane line. In addition, by processing the point cloud of the initial control point and the target path through the third-order B-spline algorithm, a smooth lane line that conforms to the actual road conditions can be generated.

[0012] In an optional implementation, based on the sampling strategy and the control point, the sampling point corresponding to the control point is determined, including: determining the B-spline curve corresponding to the target path according to the control point and the third-order B-spline algorithm; discretizing the B-spline curve to determine the sampling point corresponding to the control point.

[0013] The lane line fitting optimization method provided in this embodiment can generate a smooth curve because its basis function is a cubic polynomial, thereby ensuring that the vehicle can travel along a smooth path during driving, thereby improving driving comfort and safety. In addition, the B-spline curve has continuity between control points, so that the curve will not have abrupt turns or jumps when crossing control points, thereby improving the accuracy of lane line detection, so that the vehicle can accurately identify and follow the lane line. In addition, by discretizing the B-spline curve, a series of accurate sampling points can be obtained, and the sampling points can be further used in subsequent processing steps such as vehicle positioning and path planning to improve the accuracy of the entire system.

[0014] In an optional embodiment, the method also includes: obtaining observation points, and determining the sampling points closest to the observation points based on the observation points; determining the target parameters corresponding to each control point based on the third-order B-spline algorithm and the sampling points; determining the first loss function of the B-spline curve based on the target parameters and the control points, and minimizing the first loss function to determine the second target control point; and updating the B-spline curve based on the second target control point.

[0015] The lane line fitting optimization method provided in this embodiment determines the sampling point closest to the observation point so that the sampling point can accurately reflect the curve characteristics around the control point, thereby reducing the error caused by uneven distribution or sparseness of the sampling points and improving the accuracy of the B-spline curve description. In addition, the target parameters corresponding to the control point are calculated using the third-order B-spline algorithm and the sampling points, and the second target control point is determined by minimizing the first loss function. The position of the control point is continuously adjusted to make the B-spline curve better fit the original data or meet specific requirements, thereby improving the accuracy of the curve.

[0016] In an optional embodiment, determining a target point of a lane line of an adjacent lane of a lane line based on sampling points includes: determining a normal vector of each sampling point based on the sampling points, and extending a preset distance based on the direction of the normal vector to obtain an adjacent lane line; determining a nearest point between the sampling points and the point cloud of the adjacent lane line based on the sampling points and the point cloud of the adjacent lane line, and determining a normal vector of the nearest point; detecting whether a state between the normal vector of the nearest point and the normal vector of the sampling point satisfies a preset condition; if the state between the normal vector of the nearest point and the normal vector of the sampling point satisfies the preset condition, determining the nearest point as the target point of the lane line of the adjacent lane.

[0017] The lane line fitting optimization method provided in this embodiment can accurately identify which points actually belong to the adjacent lane lines by calculating the normal vectors of the sampling point and the points on its potential adjacent lane lines and comparing their states. Moreover, the normal vector, as a characteristic of the local surface, has good robustness to noise and small position errors. Even if there is a slight position deviation of the sampling point, as long as the direction of the normal vector is consistent, the adjacent lane lines can still be correctly identified. In addition, by narrowing the search range by extending the direction of the normal vector by a preset distance, the amount of point cloud data that needs to be processed can be significantly reduced, thereby improving the processing speed. Moreover, the comparison between normal vectors is relatively simple and the amount of calculation is small, and the nearest point that meets the preset conditions can be quickly found.

[0018] In an optional embodiment, the lane line point cloud is optimized based on the target point to obtain a first target control point, and a target lane line is generated based on the first target control point, including: determining the lane line of the adjacent lane based on the target point; determining the sampling point of the lane line based on the lane line; determining the adjacent sampling points of the lane line based on the sampling point of the lane line; constructing a second loss function based on the sampling point of the lane line and the adjacent sampling points, and constructing a third loss function based on the sampling point of the lane line; processing the second loss function and the third loss function using a gradient optimization algorithm to obtain a first target control point; determining a first target sampling point corresponding to the first target control point based on the first target control point; wherein each first target sampling point is output as a target lane line.

[0019] The lane line fitting optimization method provided in this embodiment can more accurately control the shape and position of the lane line by constructing the second loss function and the third loss function. And through the relationship between the sampling points of the lane line and its adjacent points, the gradient optimization algorithm is used to process the loss function, which can resist interference factors such as noise and occlusion to a certain extent, and improve the robustness of lane line detection. Even in complex or dynamically changing traffic environments, it can maintain a good detection effect.

[0020] In a second aspect, the present invention provides a lane line fitting optimization device, which includes: an acquisition processing module, which is used to acquire the lane line point cloud at each moment, and track and fuse each lane line point cloud to determine a lane line point cloud set; a fusion determination module, which is used to determine the target path corresponding to the lane line point cloud based on the lane line point cloud set, and determine at least one control point based on the target path; a first determination module, which is used to determine the sampling point corresponding to the control point based on the sampling strategy and the control point, and determine the target point of the lane line of the adjacent lane of the lane line based on the sampling point; a second determination module, which is used to optimize the lane line point cloud based on the target point, obtain the first target control point, and generate the target lane line based on the first target control point.

[0021] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the lane line fitting optimization method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the lane line fitting optimization method of the first aspect or any corresponding embodiment thereof.

[0023] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the lane line fitting optimization method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a schematic flow chart of a lane line fitting optimization method according to an embodiment of the present invention;

[0026] Figure 2 is a flow chart of another lane line fitting optimization method according to an embodiment of the present invention;

[0027] Figure 3 is a flow chart of another lane line fitting optimization method according to an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of target points of lane lines of adjacent lanes according to an embodiment of the present invention;

[0029] Figure 5 is a structural block diagram of a lane line fitting optimization device according to an embodiment of the present invention;

[0030] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] Based on relevant technologies, it can be known that in the assisted driving system, the deep learning-based perception module is a key component, responsible for extracting useful information from complex environments, such as lane lines, to support subsequent positioning and decision-making functions.

[0033] At present, in order to improve the accuracy and robustness of lane line detection, manual annotation and automatic learning methods are used. Specifically, the scheme first uses limited manual annotation data to train an initial deep learning network model, and then uses the model to predict a large number of new unlabeled scenes. For the prediction results, they are corrected and improved through a series of post-processing optimization steps, and then these optimized data are re-incorporated into the training set, and the model performance is gradually improved through iterative training. Among them, in the post-processing optimization stage of lane line detection, it mostly relies on the predicted 2D lane line points in the picture, and fits them in combination with the geometric constraints of parallel lane lines. This method uses mathematical algorithms (such as Hough transform, least squares method or RANSAC, etc.) to fit the lane lines with straight lines or curves to improve the accuracy and continuity of lane line detection.

[0034] However, detection and fitting are performed based on 2D lane line points in the image, ignoring the actual layout in three-dimensional space, which results in reduced detection accuracy in complex three-dimensional traffic environments. In addition, the 2D lane line fitting relies on mathematical algorithms and lacks optimization methods for three-dimensional space characteristics, which limits further improvement of detection performance.

[0035] Based on this, the present invention provides a lane line fitting optimization method, which obtains lane line point clouds at various times and constructs a lane line point cloud set using tracking and fusion technology, which provides rich three-dimensional information for lane line detection, thereby more accurately reflecting the actual position and shape of the lane line in three-dimensional space, thereby improving the detection accuracy. In addition, by determining the target path and control points, three-dimensional information is further used to guide the detection of lane lines, so that it is no longer solely dependent on parallel constraints in 2D images, but can more accurately fit lane lines in three-dimensional space. In addition, the process of optimizing the lane line point cloud through target points, obtaining the first target control point and generating the target lane line fully utilizes the advantages of three-dimensional information, and directly adjusts and improves the lane line in three-dimensional space through the optimization algorithm, thereby improving the accuracy and robustness of detection.

[0036] According to an embodiment of the present invention, an embodiment of a lane line fitting optimization method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] In this embodiment, a lane line fitting optimization method is provided, which can be used in computer devices, such as computers, servers, etc. Figure 1 is a flow chart of a lane line fitting optimization method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0038] Step S101, obtaining lane line point clouds at each time, and tracking and fusing each lane line point cloud to determine a lane line point cloud set.

[0039] The lane line point cloud can be used to represent the three-dimensional point set about the lane line position obtained by scanning the surrounding environment through sensors such as laser radar. Among them, the lane line point cloud contains the position information of the lane line in three-dimensional space. Specifically, the lane line point cloud at each time is obtained, such as: the lane line point cloud at time t, the lane line point cloud at time t+1, the lane line point cloud at time t+2, etc., which are not specifically limited here, and then the lane line point cloud is tracked and fused. Among them, tracking and fusion can be performed by Kalman filtering, particle filtering, etc., which are not limited here and can be implemented by technical personnel in this field.

[0040] More specifically, in the vision-centric Bird's Eye View (BEV) perception network, at time t, the output lane line Lt can be expressed as:

[0041] ; Where L is the lane line, i is the lane line number, and n is the total number of lane lines (n>0).

[0042] L at different times t represents the set of lane lines fused at all times. Represented as a lane line observed at time t, each lane line is represented by a discrete 3D point.

[0043] ; where m is the lane line, n i is the total number of lane lines (n i >0), i is the lane line number.

[0044] Step S102: determining a target path corresponding to the lane line point cloud based on the lane line point cloud set, and determining at least one control point based on the target path.

[0045] The lane point cloud set can be used to represent the set of lane point cloud data obtained after tracking and fusion, representing the continuity of the lane over a period of time. The target path refers to the path of the lane centerline or boundary line calculated based on the lane point cloud set, which is used to guide the driving direction of the vehicle. The control point refers to the key point selected on the target path, which is used as a reference in the subsequent sampling and lane optimization process.

[0046] Specifically, after obtaining the lane line point cloud set, the lane line point cloud set is used to use an algorithm (such as least squares method, curve fitting, etc.) to determine the center line or boundary line of the lane line as the target path, and control points are selected on the path. The target path and control points can also be determined by other methods, which are not specifically limited here and can be implemented by technical personnel in this field.

[0047] Step S103, based on the sampling strategy and the control point, determining the sampling point corresponding to the control point, and determining the target point of the lane line of the adjacent lane of the lane line based on the sampling point.

[0048] Specifically, the computer device system generates sampling points corresponding to the control points based on the sampling strategy and the control points. Then, these sampling points are used to infer or calculate the target points of the lane lines of the adjacent lanes. Among them, the sampling points corresponding to the control points can be obtained by sampling strategies such as uniform sampling and curvature-based sampling, or by other methods, which are not specifically limited here and can be implemented by those skilled in the art. The target points of the adjacent lanes can be determined by combining the lane line information in the map, and the nearest adjacent points in the adjacent lanes can be determined as the target points by determining the sampling points. The relative position relationship between the lane lines (such as parallel, intersection) can also be used to infer the target points, etc., which are not specifically limited here and can be implemented by those skilled in the art.

[0049] Step S104, optimizing the lane line point cloud based on the target point to obtain a first target control point, and generating a target lane line based on the first target control point.

[0050] The first target control point can be used to characterize the optimized control point, reflecting the actual position and shape of the lane line. Specifically, the lane line point cloud is optimized based on the previously determined target point. The purpose of the optimization is to make the lane line point cloud more accurately reflect the actual lane line position and shape. Then, the final target lane line is generated based on the optimized lane line point cloud or the first target control point. The target lane line will be used for navigation and obstacle avoidance of autonomous driving vehicles. Specifically, iterative algorithms such as gradient descent and particle swarm optimization can be used to optimize the control point position to minimize the error function. Lane line models such as straight line models and quadratic curve models can also be used to fit and optimize point cloud data. No specific limitation is made here, and it can be implemented by those skilled in the art.

[0051] The lane line fitting optimization method provided in this embodiment obtains the lane line point cloud at each moment and constructs a lane line point cloud set using tracking and fusion technology, which provides rich three-dimensional information for lane line detection, thereby more accurately reflecting the actual position and shape of the lane line in three-dimensional space, thereby improving the detection accuracy. In addition, by determining the target path and control points, the three-dimensional information is further used to guide the detection of the lane line, so that it is no longer solely dependent on the parallel constraints in the 2D image, but can more accurately fit the lane line in three-dimensional space. In addition, the process of optimizing the lane line point cloud through the target point, obtaining the first target control point and generating the target lane line fully utilizes the advantages of three-dimensional information, and directly adjusts and improves the lane line in three-dimensional space through the optimization algorithm, thereby improving the accuracy and robustness of the detection.

[0052] In this embodiment, a lane line fitting optimization method is provided, which can be used in the above-mentioned computer equipment, such as a computer, a server, etc. Figure 2is a flow chart of a lane line fitting optimization method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0053] Step S201, obtaining lane line point clouds at each time, and tracking and fusing each lane line point cloud to determine a lane line point cloud set.

[0054] Specifically, the above step S201 includes:

[0055] Step S2021, obtaining a first lane line point cloud of a first lane line at a first moment and a second lane line point cloud of a second lane line at a second moment; wherein the first moment is used to represent an initial moment, and each first lane line point cloud constitutes an initial lane line point cloud set.

[0056] The first moment can be used to represent the initial moment, that is, the time point when the lane line monitoring starts. The second moment can correspond to the first moment, and can be a time point subsequent to the first moment.

[0057] The first lane line refers to the lane boundary line detected or identified at a specific time (first moment). The first lane line may include a solid line, a dotted line, or other forms of road markings, etc., which are not specifically limited here. The second lane line refers to the lane boundary line detected or identified at a second moment. Similarly, the second lane line may include a solid line, a dotted line, or other forms of road markings.

[0058] The first lane line point cloud refers to the point cloud data obtained by scanning the first lane line through sensors such as laser radar. Point cloud data is a collection of a large number of three-dimensional coordinate points, each of which represents a position of the lane line in three-dimensional space. The second lane line point cloud refers to the point cloud data obtained by scanning the second lane line through sensors such as laser radar.

[0059] It should be noted that lane lines can be used to represent boundary lines on the road, to separate different lanes or to indicate the direction of travel.

[0060] Specifically, a vehicle-mounted lidar sensor can be used to scan the road at specific times (first moment and second moment) to obtain a first lane line point cloud of the first lane line at the first moment and a second lane line point cloud of the second lane line at the second moment. The first lane line point cloud of the first lane line at the first moment and the second lane line point cloud of the second lane line at the second moment can also be extracted from the camera image through image processing or machine learning algorithms, etc., which are not specifically limited here and can be implemented by those skilled in the art.

[0061] Step S2022: Determine the minimum distance between the first lane line and the second lane line based on the first lane line point cloud and the second lane line point cloud.

[0062] Since the lane lines are long and thin and there are occlusions, the observations of the previous and next frames may be discontinuous. Therefore, the distance between the two lane lines can be measured by the minimum distance between the first lane line and the second lane line, that is, the distance between the two lane lines can be measured by the minimum distance (Euclidean distance) between any two points. Specifically, the nearest pair of points in the two point clouds can be found by the nearest neighbor search algorithm in the point cloud processing library, and the distance between them can be calculated. The minimum distance between the first lane line and the second lane line can also be determined by applying a spatial index structure. Etc., which is not specifically limited here.

[0063] Step S2023, detecting whether the minimum distance is greater than a preset distance.

[0064] Step S2024: If the minimum distance is less than the preset distance, the first lane line point cloud and the second lane line point cloud are fused to obtain a lane line point cloud set.

[0065] Step S2025: If the minimum distance is greater than the preset distance, the second lane line point cloud is merged into the initial lane line point cloud set to obtain a lane line point cloud set.

[0066] After determining the minimum distance, the fusion method of the first lane line point cloud and the second lane line point cloud can be determined by detecting whether the minimum distance is greater than the preset distance. If the minimum distance is less than the preset distance, it is considered that the lane line at the two moments has not changed much, and it can be regarded as the performance of the same lane line at different moments. Therefore, the two point cloud data are fused to obtain a more complete and continuous lane line point cloud set. If the minimum distance is greater than the preset distance, it is considered that the lane line has changed significantly. The second lane line point cloud is added as a new lane line to the initial lane line point cloud set to reflect the actual changes in the road.

[0067] Specifically, before fusion, the first lane line point cloud and the second lane line point cloud can be preprocessed to remove unnecessary information such as noise and outliers. This can be achieved specifically through filtering algorithms (such as Gaussian filtering, median filtering, etc.). Then, the two point cloud data can be aligned to the same coordinate system through a registration method for subsequent fusion operations. Among them, the registration method can include three steps: feature extraction, feature matching, and transformation matrix calculation. Among them, feature extraction is used to extract representative geometric features from the point cloud; feature matching finds the correspondence between the two point clouds based on these features; finally, the transformation matrix (such as rotation matrix and translation vector) is calculated through an optimization algorithm to align the two point clouds. It can also be achieved in other ways, which are not specifically limited here and can be implemented by those skilled in the art.

[0068] In an application scenario, the first moment is used to initialize the first lane line set {L} that has been sensed, and the second moment is used to observe the lane line set {Lt2}, and each second lane line in {Lt2} is traversed. If the distance between the second lane line and a point in the first lane line is less than 0.5m, the second lane line is merged with the first lane line; if the distance between the second lane line and any first lane line in the first lane line set is greater than 0.5m, the second lane line is added to the first lane line set. If it is not limited to the first moment and the second moment, the lane lines at all subsequent moments can be processed in the same way.

[0069] Step S202: Determine the target path corresponding to the lane point cloud based on the lane point cloud set, and determine at least one control point based on the target path. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0070] Step S203: Based on the sampling strategy and the control point, determine the sampling point corresponding to the control point, and determine the target point of the lane line of the adjacent lane of the lane line based on the sampling point. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0071] Step S204: Optimize the lane line point cloud based on the target point to obtain a first target control point, and generate a target lane line based on the first target control point. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0072] The lane line fitting optimization method provided in this embodiment can capture the dynamic changes of the lane line in real time by acquiring the lane line point clouds at the first moment and the second moment (and more subsequent moments), and then accurately determine the relative position relationship between the first lane line and the second lane line by calculating the minimum distance between the two, so that the lane line point cloud can be more accurately fused or updated in subsequent processing.

[0073] In this embodiment, a lane line fitting optimization method is provided, which can be used in the above-mentioned computer equipment, such as a computer, a server, etc. Figure 3 is a flow chart of a lane line fitting optimization method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0074] Step S301, obtain the lane line point cloud at each time, and track and fuse each lane line point cloud to determine the lane line point cloud set. For details, please refer to Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0075] Step S302: determining a target path corresponding to the lane point cloud based on the lane point cloud set, and determining at least one control point based on the target path.

[0076] Specifically, the above step S302 includes:

[0077] Step S3021, construct and process the lane line point cloud set to obtain an undirected graph.

[0078] An undirected graph is a graph consisting of vertices (also called nodes) and edges connecting these vertices, where each edge is undirected. Specifically, the lane line point cloud set is converted into the form of an undirected graph for subsequent processing. Among them, the specific method may include: using a clustering algorithm (such as DBSCAN, K-means, etc.) to cluster the lane line point cloud set into several groups, each group representing a part of the lane line. Each cluster center or representative point in the cluster is used as a vertex of the undirected graph to construct edges according to the spatial relationship between the vertices (such as distance, direction, etc.) to form an undirected graph. It is also possible to determine the distance between a point in the lane line point cloud set and other points, and then determine the 10 points closest to the point, connect the point and the 10 points to form edges, repeat the operation on the lane line point cloud set, and obtain an undirected graph, etc., which is not specifically limited here and can be implemented by those skilled in the art.

[0079] Step S3022: downsample the lane point cloud set based on the first step to determine the target lane point cloud set.

[0080] Downsampling refers to the process of reducing the amount of data, which is used to reduce computational complexity or improve processing efficiency. The first step size refers to the step size or interval used in downsampling, which is used to determine which point cloud data points to retain. Specifically, the lane line point cloud set is downsampled using the first step size to reduce the amount of data and improve the efficiency of subsequent processing. Specific implementation methods may include: points can be uniformly selected from the lane line point cloud set as part of the target lane line point cloud set according to a fixed step size or interval. Different step sizes or strategies can also be used for downsampling according to the density or importance of the point cloud, etc., which are not specifically limited here and can be implemented by those skilled in the art.

[0081] Step S3023, using the Dijkstra algorithm to determine the shortest path between each point in the target lane line point cloud set.

[0082] The target lane line point cloud set is regarded as a graph, where the point cloud points are vertices and the distance between the point cloud points is the weight of the edge. Then the Dijkstra algorithm is used to calculate the shortest path from a source point to all other points.

[0083] Step S3024, compare the shortest paths, and take the longest path among the shortest paths as the target path.

[0084] Compare multiple shortest paths and find the longest one as the target path.

[0085] In an application scenario, the lane line point cloud set is downsampled with a step size of 0.5m to obtain a temporary sparse lane line point cloud set. Then the Dijkstra algorithm is used to find the shortest path from each point in the temporary sparse lane line point cloud set to other points, and the point with the longest path is selected from these shortest paths as the path of the lane line.

[0086] Step S3025: determine the initial control point based on the length of the target path.

[0087] Determine the location and number of initial control points based on the target path length or other related parameters (such as curvature, direction change, etc.). For example, set the minimum number of B-spline nodes to 3 and use third-order B-spline fitting. Based on the target path length, if the path is greater than 5m, add a control node every 2m.

[0088] Step S3026, based on the initial control points and the point cloud of the target path, the control points are determined using a third-order B-spline algorithm.

[0089] The third-order B-spline algorithm is used to smooth and fit data points. Among them, the B-spline curve has excellent properties such as local modifiability, geometric invariance and convex hull. Specifically, a third-order B-spline curve is constructed based on the initial control points and the point cloud of the target path. By adjusting the position and number of control points, the B-spline curve can better fit the point cloud data of the target path. After optimization, the final set of control points is obtained, which will be used in subsequent path planning or navigation tasks.

[0090] Step S303: Based on the sampling strategy and the control point, determine the sampling point corresponding to the control point, and determine the target point of the lane line of the adjacent lane of the lane line based on the sampling point. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0091] Step S304: Optimize the lane line point cloud based on the target point to obtain a first target control point, and generate a target lane line based on the first target control point. Figure 2 Step S204 of the illustrated embodiment will not be described in detail here.

[0092] The lane line fitting optimization method provided in this embodiment can significantly reduce the amount of data and the computational complexity of subsequent processing by downsampling the lane line point cloud set. At the same time, key information points are retained to improve the accuracy of the processing results. In addition, by searching the shortest path between each point in the target lane line point cloud set through the Dijkstra algorithm, the optimal path connecting the starting point and the end point, that is, the target path, can be accurately found, thereby accurately reflecting the actual direction of the lane line. In addition, by processing the point cloud of the initial control point and the target path through the third-order B-spline algorithm, a smooth lane line that conforms to the actual road conditions can be generated.

[0093] In an optional implementation, in the above step S303, based on the sampling strategy and the control point, determining the sampling point corresponding to the control point includes:

[0094] Step a1: Determine the B-spline curve corresponding to the target path based on the control points and the third-order B-spline algorithm.

[0095] Control points are used to define the key points of the shape of the B-spline curve. By adjusting the position of the control points, the shape of the B-spline curve can be changed. Select appropriate control points according to the target path. These control points should be able to reflect the main shape and characteristics of the target path. Use a third-order B-spline basis function. Among them, the B-spline basis function is a set of functions used to calculate the weights of points on the B-spline curve, and its form is related to the order. For a third-order B-spline, the basis function will involve the cubic power of t. According to the control points and the basis function, the points on the B-spline curve are calculated by linear combination. Specifically, each point on the B-spline curve is the weighted sum of the control point and its corresponding basis function value. Connect the calculated points in sequence to form a B-spline curve. This curve will roughly follow the direction of the target path, but it will be smoother and more controllable. The B-spline curve can also be determined by other means, which are not specifically limited here and can be implemented by those skilled in the art.

[0096] Step a2: discretize the B-spline curve to determine the sampling points corresponding to the control points.

[0097] Sampling points refer to points obtained during the discretization process, which can specifically represent specific positions on the B-spline curve. Specifically, according to the accuracy requirements of the target path and the limitations of computing resources, a suitable sampling interval is selected. The smaller the sampling interval, the denser the sampling points obtained, and the more accurate the description of the curve, but the amount of calculation will increase accordingly. Starting from the starting point of the B-spline curve, according to the determined sampling interval, the points on the curve are calculated and recorded in sequence as sampling points. This can be achieved by mapping the sampling interval to the parameter space of the B-spline curve and using the parametric equation of the B-spline curve to calculate the points at the corresponding positions. The obtained sampling points are output as a series of discrete data points, which can be used for subsequent analysis, processing or display.

[0098] The lane line fitting optimization method provided in this embodiment can generate a smooth curve because its basis function is a cubic polynomial, thereby ensuring that the vehicle can travel along a smooth path during driving, thereby improving driving comfort and safety. In addition, the B-spline curve has continuity between control points, so that the curve will not have abrupt turns or jumps when crossing control points, thereby improving the accuracy of lane line detection, so that the vehicle can accurately identify and follow the lane line. In addition, by discretizing the B-spline curve, a series of accurate sampling points can be obtained, and the sampling points can be further used in subsequent processing steps such as vehicle positioning and path planning to improve the accuracy of the entire system.

[0099] In an optional embodiment, the above method further includes:

[0100] Step b1, obtaining the observation point, and determining the sampling point closest to the observation point based on the observation point.

[0101] The observation point can be used to characterize the actual data point on the lane line point cloud. It can be obtained from experiments and measurements. Specifically, after determining the observation point, the sampling point closest to the observation point is determined by comparing the distance between the observation point and all sampling points. Specifically, the distance between points can be calculated by Euclidean distance or other suitable distance metrics. Among them, for each observation point, a traversal operation is performed to find the nearest sampling point. Other methods can also be used, which are not specifically limited here and can be implemented by those skilled in the art.

[0102] Step b2, based on the third-order B-spline algorithm and the sampling points, determine the target parameters corresponding to each control point.

[0103] The target parameter refers to the parameter value corresponding to the control point in the parameter space of the B-spline curve. Specifically, based on the known sampling points and the third-order B-spline algorithm, the target parameter corresponding to each control point in the parameter space of the B-spline curve is inferred. Specifically, the sampling points are fitted using the least squares method or other optimization algorithms to estimate the target parameters of the control points, and the target parameters of the control points are iteratively adjusted until the B-spline curve passes through or approaches all the sampling points.

[0104] Step b3, based on the target parameters and the control points, determine the first loss function of the B-spline curve, and minimize the first loss function to determine the second target control point.

[0105] The first loss function is a function that measures the difference between the B-spline curve and the sampling points. By minimizing this function, a better control point position can be found. Specifically, a loss function is first defined that calculates the difference between the B-spline curve (defined by the current control points and the target parameters) and the sampling points. Then, this loss function is minimized to find a new set of control points (called the second target control points). Specifically, gradient descent, Newton's method, or other optimization algorithms can be used to minimize the loss function.

[0106] Step b4: updating the B-spline curve based on the second target control point.

[0107] The basis functions and target parameters of the B-spline curve are recalculated based on the second target control point to reflect the new control point position. The updated B-spline curve will better fit the original data or meet specific shape requirements.

[0108] In an application scenario, for each control point, find its nearest sampling point, and then each sampling point can be expressed as s = A × P, where P is a 4 × 3 matrix composed of control points corresponding to s, each row is a control point, A is a 1 × 4 matrix, each column is the corresponding parameter of the corresponding row of P, the corresponding control point and the corresponding parameter can be inferred by the B-spline algorithm, and then minimize the first loss function, calculate the new control point, and update the B-spline. There are many optimization schemes, such as solving linear equations, etc., and this is not limited to any one scheme. The first loss function can be determined by the following formula:

[0109] ;in, For observation points, is the target parameter, is the control point, It is the lane line point cloud.

[0110] The lane line fitting optimization method provided in this embodiment determines the sampling point closest to the observation point so that the sampling point can accurately reflect the curve characteristics around the control point, thereby reducing the error caused by uneven distribution or sparseness of the sampling points and improving the accuracy of the B-spline curve description. In addition, the target parameters corresponding to the control point are calculated using the third-order B-spline algorithm and the sampling points, and the second target control point is determined by minimizing the first loss function. The position of the control point is continuously adjusted to make the B-spline curve better fit the original data or meet specific requirements, thereby improving the accuracy of the curve.

[0111] In an optional implementation, determining the target point of the lane line of the adjacent lane of the lane line based on the sampling point in the above step S303 includes:

[0112] Step c1, based on the sampling points, determine the normal vector of each sampling point, and extend the preset distance based on the direction of the normal vector to obtain the adjacent lane line.

[0113] The normal vector refers to the normal vector perpendicular to the tangent of the lane line corresponding to the sampling point. Specifically, local curve fitting (such as polynomial fitting, spline curve, etc.) can be used to approximate the shape of the lane line near the sampling point, and the tangent direction and normal direction of the point can be calculated. No specific limitation is made here, and it can be implemented by those skilled in the art.

[0114] The preset distance can be used to characterize the distance of the lane width. Specifically, starting from each sampling point, the preset distance is extended along the direction of its normal vector (considering both positive and negative directions to cover the lane lines on the left and right sides) to obtain a series of estimated points on adjacent lane lines.

[0115] Step c2, based on the sampling point and the point cloud of the adjacent lane line, determine the nearest point between the sampling point and the point cloud of the adjacent lane line, and determine the normal vector of the nearest point.

[0116] There are multiple points in the point cloud of the adjacent lane line. The distance between the sampling point and all points in the point cloud of the adjacent lane line is compared, and then the nearest point is determined. Then the normal vector of the nearest point is determined.

[0117] Step c3, detecting whether the state between the normal vector of the nearest point and the normal vector of the sampling point meets a preset condition.

[0118] The preset condition can be used to characterize whether the state between the normal vectors satisfies the similar direction, the angle is less than 30°, etc., which is not specifically limited here. The state between the normal vector of the nearest point and the normal vector of the sampling point can be used to characterize the angle between the normal vector of the nearest point and the normal vector of the sampling point, etc., which is not specifically limited here. Specifically, the angle between the normal vectors of the sampling point and the nearest point is first calculated, and then the angle is compared with the preset condition to determine whether the condition is met.

[0119] Step c4: if the state between the normal vector of the nearest point and the normal vector of the sampling point meets the preset condition, the nearest point is determined to be the target point of the lane line of the adjacent lane.

[0120] If the state between the normal vector of the nearest point and the normal vector of the sampling point meets the preset conditions, that is, the nearest point is regarded as a valid point (or target point) on the adjacent lane line, which can be used for further tasks such as lane line recognition, tracking or path planning.

[0121] In an application scenario, for each lane line, we sample according to the fitted B-spline. For each sampling point, we use B-spline to calculate its normal vector, walk 3.5m (3.5m is the average distance of the specified lane width) along the normal vector and the opposite direction of the normal vector, and calculate the nearest points of the sampling point and the sampling points of other lane lines respectively. If the distance between the nearest neighbors is less than 1m and the normal vectors are similar, they are considered to be matching points at similar positions of adjacent lane lines. Figure 4 As shown, the red points are sampling points and the green points are the nearest points.

[0122] The lane line fitting optimization method provided in this embodiment can accurately identify which points actually belong to the adjacent lane lines by calculating the normal vectors of the sampling point and the points on its potential adjacent lane lines and comparing their states. Moreover, the normal vector, as a characteristic of the local surface, has good robustness to noise and small position errors. Even if there is a slight position deviation of the sampling point, as long as the direction of the normal vector is consistent, the adjacent lane lines can still be correctly identified. In addition, by narrowing the search range by extending the direction of the normal vector by a preset distance, the amount of point cloud data that needs to be processed can be significantly reduced, thereby improving the processing speed. Moreover, the comparison between normal vectors is relatively simple and the amount of calculation is small, and the nearest point that meets the preset conditions can be quickly found.

[0123] In an optional implementation, the above step S304 includes:

[0124] Step d1, determining the lane lines of adjacent lanes based on the target point.

[0125] The direction of the lane line of the adjacent lane may be the direction of the normal vector, wherein the lane line is determined according to the direction of the normal vector with the target point as the midpoint or the starting point.

[0126] Step d2, based on the lane line, determine the sampling point of the lane line.

[0127] After determining the lane lines of the adjacent lanes, sampling points may be determined on the lane lines. The sampling points of the lane lines may be determined by selecting sampling points on the lane lines at even intervals, or by dynamically adjusting the density of the sampling points according to the curvature of the lane lines, etc., which are not specifically limited here and may be implemented by those skilled in the art.

[0128] Step d3, based on the sampling points of the lane line, determine the adjacent sampling points of the lane line.

[0129] A sampling point is determined from the lane line, and then sampling points adjacent to the sampling point are determined.

[0130] Step d4, constructing a second loss function based on the sampling points of the lane line and the adjacent sampling points, and constructing a third loss function based on the sampling points of the lane line.

[0131] The second loss function is used to evaluate the accuracy of the lane line position, such as the deviation from the real lane line. The third loss function is used to evaluate the smoothness of the lane line to prevent the detected lane line from fluctuating violently. Specifically, the mean square error or the mean absolute error can be used to construct the second loss function. The third loss function can also be constructed using the change in curvature or the change in the distance between adjacent sampling points.

[0132] Step d5, using a gradient optimization algorithm to process the second loss function and the third loss function to obtain a first target control point.

[0133] In each iteration, the gradient of the second loss function and the third loss function as a whole with respect to the model parameters needs to be calculated. The gradient represents the direction and rate at which the loss function changes with the model parameters. Specifically, it involves taking the derivative of the loss function and using the chain rule (especially important in neural networks) to calculate the gradient. The model parameters are updated using the gradient information to reduce the value of the loss function. When the value of the loss function is below a certain threshold (such as the F value), the gradient is close to zero, or the number of iterations is reached, the first target control point is obtained.

[0134] Step d6, based on the first target control point, determining a first target sampling point corresponding to the first target control point; wherein each first target sampling point is output as a target lane line.

[0135] After determining the first target control point, the corresponding first target sampling point can be determined. The first target sampling point can be determined by selecting sampling points on the lane line at uniform intervals, or by dynamically adjusting the density of the sampling points according to the curvature change of the lane line, etc., which is not specifically limited here and can be implemented by those skilled in the art. After determining all the first target sampling points, the first target sampling points are connected to form a smooth curve, namely the target lane line. Specifically, the connection method can be a simple linear interpolation or a more complex curve fitting algorithm, etc., which is not specifically limited here and can be implemented by those skilled in the art.

[0136] In an application scenario, for lane lines L i , L j The sampling point and sampling points , determine the next sampling point and Are the normal vectors of the four points similar? If so, the second loss function can be constructed. The target parameters of the B-spline corresponding to the four points are , , , The corresponding control point is , , , ; Among them, the second loss function is:

[0137] ;in,

[0138] ||_1 is the 1-norm, and ||_2 is the 2-norm, which is the Euclidean distance.

[0139] In addition, another constraint needs to be added to constrain the change range of the control points of the lane line B-spline. i Each sampling point s on i , the sampling point s can be s i = A i ×P i ; Among them, the third loss function is:

[0140] .

[0141] The lane line fitting optimization method provided in this embodiment can more accurately control the shape and position of the lane line by constructing the second loss function and the third loss function. And through the relationship between the sampling points of the lane line and its adjacent points, the gradient optimization algorithm is used to process the loss function, which can resist interference factors such as noise and occlusion to a certain extent, and improve the robustness of lane line detection. Even in complex or dynamically changing traffic environments, it can maintain a good detection effect.

[0142] In this embodiment, a lane line fitting optimization device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0143] This embodiment provides a lane line fitting optimization device, such as Figure 5 As shown, it includes: an acquisition processing module 501, which is used to acquire the lane line point cloud at each moment, and track and fuse each lane line point cloud to determine the lane line point cloud set; a fusion determination module 502, which is used to determine the target path corresponding to the lane line point cloud based on the lane line point cloud set, and determine at least one control point based on the target path; a first determination module 503, which is used to determine the sampling point corresponding to the control point based on the sampling strategy and the control point, and determine the target point of the lane line of the adjacent lane of the lane line based on the sampling point; a second determination module 504, which is used to optimize the lane line point cloud based on the target point, obtain the first target control point, and generate the target lane line based on the first target control point.

[0144] In an optional embodiment, the acquisition processing module 501 includes: a first acquisition unit, used to acquire a first lane line point cloud of a first lane line at a first moment and a second lane line point cloud of a second lane line at a second moment; wherein the first moment is used to represent the initial moment, and each first lane line point cloud constitutes an initial lane line point cloud set; a first determination unit, used to determine the minimum distance between the first lane line and the second lane line based on the first lane line point cloud and the second lane line point cloud; a first detection unit, used to detect whether the minimum distance is greater than a preset distance; a first fusion unit, used to fuse the first lane line point cloud and the second lane line point cloud to obtain a lane line point cloud set if the minimum distance is less than the preset distance; and a second fusion unit, used to merge the second lane line point cloud into the initial lane line point cloud set to obtain a lane line point cloud set if the minimum distance is greater than the preset distance.

[0145] In an optional embodiment, the fusion determination module 502 includes: a first processing unit, which is used to construct and process the lane line point cloud set to obtain an undirected graph; a second determination unit, which is used to downsample the lane line point cloud set based on the first step length to determine the target lane line point cloud set; a third determination unit, which is used to use the Dijkstra algorithm to determine the shortest path between each point in the target lane line point cloud set; a fourth determination unit, which is used to compare the shortest paths and take the longest path in the shortest paths as the target path; a fifth determination unit, which is used to determine the initial control point based on the length of the target path; and a sixth determination unit, which is used to determine the control point based on the initial control point and the point cloud of the target path using a third-order B-spline algorithm.

[0146] In an optional embodiment, the first determination module 503 includes: a seventh determination unit, used to determine the B-spline curve corresponding to the target path according to the control points and the third-order B-spline algorithm; an eighth determination unit, used to discretize the B-spline curve to determine the sampling points corresponding to the control points.

[0147] In an optional embodiment, the above-mentioned device also includes: an acquisition and determination module, which is used to acquire observation points and determine the sampling points closest to the observation points based on the observation points; a target parameter determination module, which is used to determine the target parameters corresponding to each control point based on the third-order B-spline algorithm and the sampling points; a second target control point determination module, which is used to determine the first loss function of the B-spline curve based on the target parameters and the control points, and minimize the first loss function to determine the second target control point; an update module, which is used to update the B-spline curve based on the second target control point.

[0148] In an optional embodiment, the first determination module 503 includes: a ninth determination unit, which is used to determine the normal vector of each sampling point based on the sampling point, and to extend a preset distance based on the direction of the normal vector to obtain an adjacent lane line; a tenth determination unit, which is used to determine the nearest point between the sampling points and the point cloud of the adjacent lane line based on the sampling point and the point cloud of the adjacent lane line, and to determine the normal vector of the nearest point; a second detection unit, which is used to detect whether the state between the normal vector of the nearest point and the normal vector of the sampling point meets a preset condition; and a determination unit, which is used to determine that the nearest point is the target point of the lane line of the adjacent lane if the state between the normal vector of the nearest point and the normal vector of the sampling point meets the preset condition.

[0149] In an optional embodiment, the second determination module 504 includes: an eleventh determination unit, used to determine the lane line of the adjacent lane based on the target point; a twelfth determination unit, used to determine the sampling point of the lane line based on the lane line; a thirteenth determination unit, used to determine the adjacent sampling points of the lane line based on the sampling point of the lane line; a first construction unit, used to construct a second loss function based on the sampling point of the lane line and the adjacent sampling points, and to construct a third loss function based on the sampling point of the lane line; a second construction unit, used to process the second loss function and the third loss function using a gradient optimization algorithm to obtain a first target control point; a fourteenth determination unit, used to determine the first target sampling point corresponding to the first target control point based on the first target control point; wherein each first target sampling point is output as a target lane line.

[0150] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0151] The lane line fitting optimization device in this embodiment is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0152] The embodiment of the present invention also provides a computer device having the above Figure 5 The lane line fitting optimization device shown.

[0153] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0154] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0155] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0156] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0158] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0159] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0160] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0161] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A lane line fitting optimization method, characterized in that: The method comprises: Obtaining lane line point clouds at each time, and tracking and fusing each lane line point cloud to determine a lane line point cloud set; Determine a target path corresponding to the lane point cloud based on the lane point cloud set, and determine at least one control point based on the target path; Based on the sampling strategy and the control point, determining a sampling point corresponding to the control point, and determining a target point of a lane line of an adjacent lane of the lane line based on the sampling point; Optimizing the lane line point cloud based on the target point to obtain a first target control point, and generating a target lane line based on the first target control point; The obtaining of lane line point clouds at each time, tracking and fusing each lane line point cloud, and determining a lane line point cloud set includes: Obtain a first lane line point cloud of a first lane line at a first moment and a second lane line point cloud of a second lane line at a second moment; wherein the first moment is used to represent an initial moment, and each first lane line point cloud constitutes an initial lane line point cloud set; Determine a minimum distance between the first lane line and the second lane line based on the first lane line point cloud and the second lane line point cloud; Detecting whether the minimum distance is greater than a preset distance; If the minimum distance is less than the preset distance, the first lane line point cloud and the second lane line point cloud are merged to obtain a lane line point cloud set; If the minimum distance is greater than the preset distance, merging the second lane line point cloud into the initial lane line point cloud set to obtain a lane line point cloud set; The step of determining a target point of a lane line of an adjacent lane of the lane line based on the sampling point comprises: Based on the sampling points, determine the normal vector of each sampling point, and extend a preset distance based on the direction of the normal vector to obtain an adjacent lane line; Based on the sampling point and the point cloud of the adjacent lane line, determine the nearest point between the sampling point and the point cloud of the adjacent lane line, and determine the normal vector of the nearest point; Detecting whether a state between the normal vector of the nearest point and the normal vector of the sampling point satisfies a preset condition; If the state between the normal vector of the nearest point and the normal vector of the sampling point meets a preset condition, determining that the nearest point is the target point of the lane line of the adjacent lane; The optimizing the lane line point cloud based on the target point to obtain a first target control point, and generating a target lane line based on the first target control point includes: Determine a lane line of an adjacent lane based on the target point; Based on the lane line, determining a sampling point of the lane line; Based on the sampling point of the lane line, determining adjacent sampling points of the lane line; Based on the sampling point of the lane line and the adjacent sampling points, construct a second loss function, and based on the sampling point of the lane line, construct a third loss function; Processing the second loss function and the third loss function using a gradient optimization algorithm to obtain a first target control point; Based on the first target control point, a first target sampling point corresponding to the first target control point is determined; wherein each of the first target sampling points is output as the target lane line.

2. The lane line fitting optimization method according to claim 1, characterized in that: The determining, based on the lane line point cloud set, a target path corresponding to the lane line point cloud, and determining at least one control point based on the target path, includes: Constructing and processing the lane line point cloud set to obtain an undirected graph; Downsampling the lane point cloud set based on the first step length to determine a target lane point cloud set; The Dijkstra algorithm is used to determine the shortest path between each point in the target lane line point cloud set; Comparing the shortest paths, and taking the longest path among the shortest paths as the target path; Determining an initial control point based on the length of the target path; Based on the initial control points and the point cloud of the target path, the control points are determined using a third-order B-spline algorithm.

3. The lane line fitting optimization method according to claim 2, characterized in that: The step of determining the sampling point corresponding to the control point based on the sampling strategy and the control point includes: Determine a B-spline curve corresponding to the target path according to the control points and a third-order B-spline algorithm; Discretization is performed on the B-spline curve to determine sampling points corresponding to the control points.

4. The lane line fitting optimization method according to claim 3, characterized in that: The method further comprises: Acquire an observation point, and determine a sampling point closest to the observation point based on the observation point; Based on the third-order B-spline algorithm and the sampling points, determining the target parameters corresponding to each of the control points; Based on the target parameter and the control point, determine a first loss function of the B-spline curve, and minimize the first loss function to determine a second target control point; The B-spline curve is updated based on the second target control point.

5. A lane line fitting optimization device, characterized in that: The device comprises: An acquisition processing module is used to acquire lane line point clouds at various times, and to track and fuse each lane line point cloud to determine a lane line point cloud set; A fusion determination module, configured to determine a target path corresponding to the lane line point cloud based on the lane line point cloud set, and determine at least one control point based on the target path; A first determination module, configured to determine a sampling point corresponding to the control point based on a sampling strategy and the control point, and determine a target point of a lane line of an adjacent lane of the lane line based on the sampling point; A second determination module is used to optimize the lane line point cloud based on the target point to obtain a first target control point, and generate a target lane line based on the first target control point; The acquisition processing module includes: a first acquisition unit, which is used to acquire a first lane line point cloud of a first lane line at a first moment and a second lane line point cloud of a second lane line at a second moment; wherein the first moment is used to represent an initial moment, and each first lane line point cloud constitutes an initial lane line point cloud set; a first determination unit, which is used to determine a minimum distance between the first lane line and the second lane line based on the first lane line point cloud and the second lane line point cloud; a first detection unit, which is used to detect whether the minimum distance is greater than a preset distance; a first fusion unit, which is used to fuse the first lane line point cloud and the second lane line point cloud to obtain a lane line point cloud set if the minimum distance is less than the preset distance; and a second fusion unit, which is used to merge the second lane line point cloud into the initial lane line point cloud set to obtain a lane line point cloud set if the minimum distance is greater than the preset distance; The first determination module includes: a ninth determination unit, which is used to determine the normal vector of each sampling point based on the sampling point, and to obtain the adjacent lane line by extending the preset distance based on the direction of the normal vector; a tenth determination unit, which is used to determine the nearest point between the sampling point and the point cloud of the adjacent lane line based on the sampling point and the point cloud of the adjacent lane line, and to determine the normal vector of the nearest point; a second detection unit, which is used to detect whether the state between the normal vector of the nearest point and the normal vector of the sampling point meets the preset condition; a determination unit, which is used to determine that the nearest point is the target point of the lane line of the adjacent lane if the state between the normal vector of the nearest point and the normal vector of the sampling point meets the preset condition; The second determination module includes: an eleventh determination unit, used to determine the lane line of the adjacent lane based on the target point; a twelfth determination unit, used to determine the sampling point of the lane line based on the lane line; a thirteenth determination unit, used to determine the adjacent sampling points of the lane line based on the sampling point of the lane line; a first construction unit, used to construct a second loss function based on the sampling point of the lane line and the adjacent sampling points, and to construct a third loss function based on the sampling point of the lane line; a second construction unit, used to process the second loss function and the third loss function using a gradient optimization algorithm to obtain a first target control point; a fourteenth determination unit, used to determine a first target sampling point corresponding to the first target control point based on the first target control point; wherein each first target sampling point is output as a target lane line.

6. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the lane line fitting optimization method according to any one of claims 1 to 4 by executing the computer instructions.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the lane line fitting optimization method according to any one of claims 1 to 4.