Lane segmentation modeling and tracking method based on filtering and optimization

Abstract

The present invention relates to a method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization, including: based on the lane line update point set of the previous frame, combined with the odometer information of the current vehicle, the lane line update point set of the previous frame is transformed to the current frame self-vehicle coordinate system to obtain a prediction point set; the lane line prediction point set is associated with the lane line detection point set of the current frame; based on the association result, the lane line track is managed and the start and end points of the lane line are calculated; based on the start and end points of the lane line track, the lane line segmentation modeling is performed; an optimization problem is constructed for solution; based on the optimization result, the update point set and corresponding variance of each lane line are calculated. The present invention also relates to a corresponding device, a processor and a storage medium thereof. The method, device, processor and storage medium of the present invention are adopted to effectively avoid the storage and maintenance of historical multi-frame data while ensuring tracking stability, thereby improving the efficiency and real-time performance of the algorithm.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and in particular to the field of lane modeling technology, and specifically refers to a method, device, processor and computer-readable storage medium thereof for lane line segmentation modeling and tracking processing based on a combination of filtering and optimization. Background Art

[0002] Lane line detection and tracking is one of the essential links to realize intelligent driving functions. Common functions such as lane keeping and lane departure warning are significantly dependent on continuous and stable lane line tracking results; in high-level intelligent driving solutions, the relationship between lane lines and targets is also relied on to generate the decision attributes of the corresponding targets. At present, lane line detection is generally achieved through deep learning. The deep learning model can output a single-frame lane line detection point, and lane line tracking is based on the single-frame lane line detection point, combined with the vehicle's motion state, and multi-frame processing is performed to finally output a stable and smooth lane line perception result.

[0003] The patent application with application number CN202311088049.2 proposes a lane tracking method. This method uses spline curves to model lane lines, predicts the filter state quantity according to the forward and backward states of the vehicle, and updates it with lane detection points. This method depends on whether the selected spline curve modeling is reasonable. The fitting effect of a single spline curve is poor in scenes such as intersections; this method is based on filtering implementation and can only track each lane line independently, and cannot handle the mutual constraints between lanes.

[0004] The patent application with application number CN202410241729.1 proposes a lane tracking method based on graph optimization. Based on positioning information, this method transforms the historical lane detection results into the current frame vehicle coordinate system, and establishes an optimization problem to solve it. Under the premise that enough historical lane detection results are retained, this method can obtain the global optimum, but the computational complexity is high and does not meet the real-time requirements; if only the lane detection results of two adjacent frames are retained, the optimization results are easily affected by abnormal detection and the tracking results are unstable.

[0005] Based on this, it is necessary to propose an optimization solution to solve the defects and shortcomings of the existing technology. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method, device, processor and computer-readable storage medium for lane line segmentation modeling and tracking processing based on a combination of filtering and optimization.

[0007] In order to achieve the above-mentioned purpose, the method, device, processor and computer-readable storage medium of lane segmentation modeling and tracking processing based on the combination of filtering and optimization of the present invention are as follows:

[0008] The method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization has the following main features:

[0009] (1) Based on the lane line update point set of the previous frame and the odometer information of the current vehicle, the lane line update point set of the previous frame is transformed into the current frame ego vehicle coordinate system to obtain the prediction point set, thus completing the prediction processing for the lane line of the previous frame;

[0010] (2) Associating the lane line prediction point set with the lane line detection point set of the current frame to construct an association cost matrix;

[0011] (3) Based on the association results, the lane line trajectory is managed and the start and end points of the lane line are calculated to determine the range to be optimized;

[0012] (4) Based on the start and end points of the lane line trajectory, lane line segmented cubic polynomial curve modeling is performed;

[0013] (5) Using the factor graph to construct an optimization problem and solve it to obtain the modeling parameters of each lane segment;

[0014] (6) Based on the optimization results, the update point set and corresponding variance of each lane line are calculated to complete the track update.

[0015] Preferably, the step (1) is specifically as follows:

[0016] Get the displacement vector of the lane lines in two adjacent frames based on the odometer information of the current vehicle And the corresponding corner , and construct the rotation transformation relationship as follows:

[0017] ;

[0018] in, Update the points in the point set for the previous frame, is the corresponding prediction point;

[0019] All the points in the lane line update point set of the previous frame are transformed to the current coordinate system according to the above formula to obtain the predicted point set of the current frame.

[0020] Preferably, the step (2) specifically includes the following steps:

[0021] (2.1) For each lane line prediction point set, calculate the distance between it and each lane line detection point set, so as to construct the association cost matrix;

[0022] (2.2) Use the Hungarian algorithm to find the matching result with the minimum total association cost.

[0023] Preferably, the step (2.1) specifically includes the following steps:

[0024] (2.1.1) First, obtain the intersection range of the prediction point set and the detection point set in the x direction, then calculate the area enclosed by the two point sets in the x intersection range, and finally divide the area by the length of the intersection range to obtain the integrated average distance;

[0025] (2.1.2) After calculating the integrated average distance between each prediction point set and each detection point set, fill it into the corresponding rows and columns of the associated cost matrix to complete the matrix construction.

[0026] Preferably, the step (3) specifically comprises the following steps:

[0027] (3.1) For the unassociated detection point set, a new track id is assigned to it as a new lane line track. For the new lane line track, the starting and ending points of the detection point set are the starting and ending points of the lane line;

[0028] (3.2) For the associated lane line trajectory, the starting endpoint of the lane line is selected from the closest point of the predicted point set and the closest point of the detection point set, and the ending endpoint of the lane line is selected from the farthest point of the predicted point set and the farthest point of the detection point set, whichever is farther.

[0029] (3.3) For unassociated lane line trajectories, the starting and ending points of the predicted point set are the starting and ending points of the lane line.

[0030] Preferably, the step (4) will segment the lane lines according to the starting and ending points of each lane line track, specifically:

[0031] The starting and ending points are and , divided into a section according to the preset interval s in the x direction, then [ , ) is a paragraph, [ , ) is the next segment, and so on; for the last segment whose length does not meet the interval s, that is, , where n is the current cumulative number of segments, the last segment range is [ , ) ;

[0032] Each lane line is represented by a cubic polynomial curve Modeling is performed, where , , , is the curve coefficient, that is, the variable to be optimized, and y is the y-axis coordinate of any point on the curve.

[0033] Preferably, the step (5) specifically comprises the following steps:

[0034] (5.1) constructing scattered point constraints, lane segment constraints and lane segment constraints based on the prediction point set and the detection point set respectively;

[0035] (5.2) Combining the above constraints, the factor graph is used to construct an optimization problem to solve it, so as to obtain the cubic polynomial curve coefficients of each lane segment.

[0036] Preferably, the step (5.1) constructs the scattered point constraint in the following manner:

[0037] According to the coordinates of each scattered point , assign it to the corresponding lane segment. Specifically, if , then the scattered point is assigned to the nth lane line, and the residual from the scattered point to the corresponding lane line segment cubic polynomial curve is calculated in the following way to construct the scattered point factor :

[0038] ;

[0039] in, is the cubic curve coefficient corresponding to the nth lane line.

[0040] Preferably, the step (5.1) constructs the lane segment constraints in the following manner:

[0041] ;

[0042] in, is the intra-segment constraint factor;

[0043] The step (5.1) described above constructs the lane segment constraints in the following way:

[0044] = ;

[0045] in, is the cubic curve coefficient corresponding to the n-1th lane line, is the cubic curve coefficient corresponding to the nth lane line,

[0046] ;

[0047] is the inter-segment constraint coefficient matrix, The x value that separates the two segments.

[0048] Preferably, the step (6) specifically includes the following steps:

[0049] (6.1) The cubic curve coefficient optimization results of each lane segment are sampled equidistantly at fixed intervals in the x direction, and all sampling points are combined to obtain the update point set of the corresponding lane line;

[0050] (6.2) For the associated track, for each point in the update point set, traverse the corresponding prediction point set and detection point set respectively, find the nearest prediction point and detection point, and use Kalman filtering to calculate the variance of the update point:

[0051] ;

[0052] in, is the updated variance, is the variance of the nearest prediction point, is the variance of the nearest detection point;

[0053] For unassociated tracks, ; For new tracks, .

[0054] The device for lane segmentation modeling and tracking processing based on the combination of filtering and optimization, wherein the device comprises:

[0055] a processor configured to execute computer executable instructions;

[0056] A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization are implemented.

[0057] The processor for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization are implemented.

[0058] The computer-readable storage medium stores a computer program thereon, and the computer program can be executed by a processor to implement the steps of the above-mentioned method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization.

[0059] The method, device, processor and computer-readable storage medium of lane line segmentation modeling and tracking processing based on the combination of filtering and optimization of the present invention are adopted. Lane line segmentation modeling can improve the disadvantage of insufficient characterization ability in single-segment curve modeling and improve the scene generalization ability; based on the hybrid strategy of filtering and optimization, historical frame information and its uncertainty can be transmitted in a recursive manner, while ensuring tracking stability, effectively avoiding the storage and maintenance of historical multi-frame data, improving the efficiency and real-time performance of the algorithm, and having certain practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 The present invention is a flow chart of the lane segmentation modeling and tracking method based on the combination of filtering and optimization.

[0061] Figure 2 The present invention uses a factor graph to model a single lane line schematic diagram. DETAILED DESCRIPTION

[0062] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific embodiments.

[0063] Before describing in detail embodiments according to the present invention, it should be noted that, hereinafter, the terms "comprises", "includes" or any other variations are intended to cover non-exclusive inclusion, whereby a process, method, article or apparatus comprising a series of elements includes not only these elements, but also other elements not explicitly listed or inherent to such process, method, article or apparatus.

[0064] Before explaining this technical solution in detail, the following agreements are made on relevant technical terms:

[0065] 1. Lane detection point set: refers to the lane detection result of the current frame input. Each detection point set corresponds to the detection result of a lane, and each point contains the noise variance information of the detection.

[0066] 2. Lane line update point set: refers to the lane line result obtained after the current frame is filtered and optimized. Each update point set corresponds to the tracking result of a lane line, and each point contains the updated variance information.

[0067] 3. Lane line prediction point set: refers to the result of predicting the lane line update point set of the previous frame to the current frame. Each prediction point set corresponds to the prediction result of a lane line, and each point contains the variance information after prediction.

[0068] See also Figure 1 As shown, the method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization, wherein the method comprises the following steps:

[0069] (1) Based on the lane line update point set of the previous frame and the odometer information of the current vehicle, the lane line update point set of the previous frame is transformed into the current frame ego vehicle coordinate system to obtain the prediction point set, thus completing the prediction processing for the lane line of the previous frame;

[0070] (2) Associating the lane line prediction point set with the lane line detection point set of the current frame to construct an association cost matrix;

[0071] (3) Based on the association results, the lane line trajectory is managed and the start and end points of the lane line are calculated to determine the range to be optimized;

[0072] (4) Based on the start and end points of the lane line trajectory, lane line segmented cubic polynomial curve modeling is performed;

[0073] (5) Using the factor graph to construct an optimization problem and solve it to obtain the modeling parameters of each lane segment;

[0074] (6) Based on the optimization results, the update point set and corresponding variance of each lane line are calculated to complete the track update.

[0075] As a preferred embodiment of the present invention, the step (1) is specifically as follows:

[0076] Get the displacement vector of the lane lines in two adjacent frames based on the odometer information of the current vehicle And the corresponding corner , and construct the rotation transformation relationship as follows:

[0077] ;

[0078] in, Update the points in the point set for the previous frame, is the corresponding prediction point;

[0079] All the points in the lane line update point set of the previous frame are transformed to the current coordinate system according to the above formula to obtain the predicted point set of the current frame.

[0080] As a preferred embodiment of the present invention, the step (2) specifically includes the following steps:

[0081] (2.1) For each lane line prediction point set, calculate the distance between it and each lane line detection point set, so as to construct the association cost matrix;

[0082] (2.2) Use the Hungarian algorithm to find the matching result with the minimum total association cost.

[0083] As a preferred embodiment of the present invention, the step (2.1) specifically includes the following steps:

[0084] (2.1.1) First, obtain the intersection range of the prediction point set and the detection point set in the x direction, then calculate the area enclosed by the two point sets in the x intersection range, and finally divide the area by the length of the intersection range to obtain the integrated average distance;

[0085] (2.1.2) After calculating the integrated average distance between each prediction point set and each detection point set, fill it into the corresponding rows and columns of the associated cost matrix to complete the matrix construction.

[0086] As a preferred embodiment of the present invention, the step (3) specifically includes the following steps:

[0087] (3.1) For the unassociated detection point set, a new track id is assigned to it as a new lane line track. For the new lane line track, the starting and ending points of the detection point set are the starting and ending points of the lane line;

[0088] (3.2) For the associated lane line trajectory, the starting endpoint of the lane line is selected from the closest point of the predicted point set and the closest point of the detection point set, and the ending endpoint of the lane line is selected from the farthest point of the predicted point set and the farthest point of the detection point set, whichever is farther.

[0089] (3.3) For unassociated lane line trajectories, the starting and ending points of the predicted point set are the starting and ending points of the lane line.

[0090] As a preferred embodiment of the present invention, the step (4) is to segment the lane lines according to the starting and ending points of each lane line track, specifically:

[0091] The starting and ending points are and , divided into a section according to the preset interval s in the x direction, then [ , ) is a paragraph, [ , ) is the next segment, and so on; for the last segment whose length does not meet the interval s, that is, , where n is the current cumulative number of segments, the last segment range is [ , ) ;

[0092] Each lane line is represented by a cubic polynomial curve Modeling is performed, where , , , is the curve coefficient, that is, the variable to be optimized, and y is the y-axis coordinate of any point on the curve.

[0093] As a preferred embodiment of the present invention, the step (5) specifically includes the following steps:

[0094] (5.1) constructing scattered point constraints, lane segment constraints and lane segment constraints based on the prediction point set and the detection point set respectively;

[0095] (5.2) Combining the above constraints, the factor graph is used to construct an optimization problem to solve it, so as to obtain the cubic polynomial curve coefficients of each lane segment.

[0096] As a preferred embodiment of the present invention, the step (5.1) constructs the scattered point constraint in the following manner:

[0097] According to the coordinates of each scattered point , assign it to the corresponding lane segment. Specifically, if , then the scattered point is assigned to the nth lane line, and the residual from the scattered point to the corresponding lane line segment cubic polynomial curve is calculated in the following way to construct the scattered point factor :

[0098] ;

[0099] in, is the cubic curve coefficient corresponding to the nth lane line.

[0100] As a preferred embodiment of the present invention, the step (5.1) constructs the lane segment constraints in the following manner:

[0101] ;

[0102] in, is the intra-segment constraint factor;

[0103] The step (5.1) described above constructs the lane segment constraints in the following way:

[0104] = ;

[0105] in, is the cubic curve coefficient corresponding to the n-1th lane line, is the cubic curve coefficient corresponding to the nth lane line,

[0106] ;

[0107] is the inter-segment constraint coefficient matrix, The x value that separates the two segments.

[0108] As a preferred embodiment of the present invention, the step (6) specifically includes the following steps:

[0109] (6.1) The cubic curve coefficient optimization results of each lane segment are sampled equidistantly at fixed intervals in the x direction, and all sampling points are combined to obtain the update point set of the corresponding lane line;

[0110] (6.2) For the associated track, for each point in the update point set, traverse the corresponding prediction point set and detection point set respectively, find the nearest prediction point and detection point, and use Kalman filtering to calculate the variance of the update point:

[0111] ;

[0112] in, is the updated variance, is the variance of the nearest prediction point, is the variance of the nearest detection point;

[0113] For unassociated tracks, ; For new tracks, .

[0114] In practical application, the relevant processing steps involved in this technical solution are as follows:

[0115] 1. Prediction of lane lines in the previous frame. Based on the lane line update point set in the previous frame and the odometer information of the current vehicle, the lane line update point set in the previous frame is transformed to the current frame vehicle coordinate system to obtain the predicted point set. At the same time, the uncertainty of the prediction process is considered and the variance of the predicted points is increased accordingly.

[0116] 2. Associate the lane line prediction point set with the lane line detection point set of the current frame. For each lane line prediction point set, calculate the distance between it and each lane line detection point set and construct an association cost matrix. The distance between the two point sets can be measured by Euclidean distance, Hausdorff distance, Fréchet distance, dynamic time warping, etc. Use the Hungarian algorithm to solve the matching result with the minimum total association cost.

[0117] 3. Manage lane line tracks based on the association results and calculate the start and end points of the lane lines. For the associated lane line tracks, it has both the prediction point set and the detection point set; for the unassociated lane line tracks, it only has the prediction point set; for the unassociated detection point set, a new lane line track is established, and the new lane line track only has the detection point set. For all lane line tracks, calculate the start and end point positions according to the point sets they have, and determine the range to be optimized.

[0118] 4. Based on the start and end points of the lane line trajectory, perform lane line segmentation modeling and construct an optimization problem for solution. The specific steps of constructing the optimization problem are as follows:

[0119] 4.1 Construct scattered point constraints based on the prediction point set and the detection point set. For the associated lane line track, the prediction point set and the detection point set are assigned to the corresponding lane line segment, and the distance from the point to the lane line segment modeling curve is calculated to construct the constraint; for the unassociated lane line track or the new track, it is only necessary to assign the prediction point set or the detection point set to the corresponding lane line segment and construct the constraint. The uncertainty of the scattered point constraint is represented by the variance of the point.

[0120] 4.2 Constructing constraints between lane segments. Different lane segments are combined to represent a complete lane line, so different lane segments need to comply with constraints such as continuity and smoothness. According to different segmentation and modeling methods, construct corresponding constraints between lane segments.

[0121] 4.3 Constructing constraints within lane segments. In general, the length of each lane segment is relatively small. In actual scenarios, a single lane segment should be relatively flat. Therefore, a flat constraint can be constructed for each lane segment to minimize the changes in the orientation angle, curvature, etc. of the lane segment. This constraint is optional.

[0122] 4.4 Constructing a constraint between lane lines. In actual scenarios, the lane lines on the same road are parallel, so a parallel constraint between lane lines can be constructed. This constraint is optional.

[0123] 4.5 Combining the above constraints, we can construct an optimization problem. We can use gradient descent method, Gauss-Newton method and other solving algorithms to solve the modeling parameters of each lane segment.

[0124] 5. Based on the optimization results, calculate the updated point set and corresponding variance of each lane line. The specific steps are as follows:

[0125] 5.1 Based on the optimized modeling parameters of each lane segment, sampling in a certain way can obtain a series of sampling points of the corresponding segment. The sampling points of different lane segments of the same lane line are combined together to form the update point set of a single lane line.

[0126] 5.2 For the associated lane track, update the variance of the points in the update point set. First, determine the prediction point and detection point corresponding to the update point. Then, based on the variance of the latter two, the Kalman filter variance update formula can be used to directly calculate the variance of the update point; for unassociated tracks or new tracks, the variance of the update point is the variance of the corresponding prediction point or detection point.

[0127] The technical solution will be further described below in conjunction with specific implementation methods:

[0128] Step 1: Prediction of lane lines in the previous frame. The displacement vector of lane lines in two adjacent frames can be obtained based on the odometer information of the vehicle. And the corresponding corner , based on these three, a rotation transformation relationship can be constructed:

[0129] ;

[0130] in, Update the points in the point set for the previous frame, is the corresponding prediction point. All the points in the lane line update point set of the previous frame are transformed to the current coordinate system according to the above formula to obtain the prediction point set of the current frame. Since the measurement of the vehicle odometer also has errors, the influence of this measurement noise needs to be added to the variance of the prediction point set.

[0131] Step 2: Associate the lane line prediction point set with the lane line detection point set:

[0132] 2.1 For each lane line prediction point set, calculate the distance between it and each lane line detection point set, and construct an associated cost matrix. In this embodiment, the integral average distance is used to measure similarity. Specifically, first obtain the intersection range of the two point sets in the x direction, that is, take the larger minimum x of the two point sets and the smaller maximum x; then calculate the area enclosed by the two point sets in this x intersection range; finally, divide the area by the length of the intersection range to obtain the integral average distance. After calculating the integral average distance for each prediction point set and each detection point set, fill in the corresponding rows and columns of the associated cost matrix to complete the matrix construction. Specifically, for the prediction point set i and the detection point set j, the integral average distance between them is recorded as , then the element value of the i-th row and j-th column of the cost matrix is .

[0133] 2.2 Use the Hungarian algorithm to find the matching result with the minimum total association cost.

[0134] Step 3: Track management and start and end point calculation. For unassociated detection point sets, assign them new track ids as new lane line tracks. For associated lane line tracks, they have both prediction point sets and measurement point sets, so the starting endpoint is the closer point between the nearest point of the prediction point set and the nearest point of the detection point set, and the ending endpoint is the farther point between the farthest point of the prediction point set and the farthest point of the detection point set; for unassociated lane line tracks, they only have prediction point sets, so the start and end points of the prediction point set are the start and end points; new lane line tracks only have detection point sets, so the start and end points of the detection point set are the start and end points.

[0135] Step 4: Use piecewise cubic polynomial curves to model lane lines. Segment the lane lines according to the starting and ending points of each lane line’s trajectory. Specifically, the starting and ending points are and In this embodiment, the x-axis is divided into sections at intervals of 10 m, then [ , ) is a paragraph, [ , ) is a section, and so on. If the length of the last section does not meet 10 m, that is, , where n is the current cumulative number of segments, the last segment range is [ , ). Each lane line is represented by a cubic polynomial curve Modeling is performed, where , , , is the curve coefficient, that is, the variable to be optimized.

[0136] Step 5: Construct the optimization problem. In this embodiment, the optimization problem is modeled using a factor graph. Figure 2 As shown in FIG, the factor graph modeling result of a single lane line is shown. In the figure, the circle represents the variable to be optimized, which is the corresponding cubic polynomial coefficient in this embodiment. ; Rectangles represent constraint factors, including scatter factor, intra-segment constraint factor and inter-segment constraint factor. The first two are unary factors, which are only related to a single optimization variable. The inter-segment constraint factor is a binary factor, which represents the constraint between two optimization variables. In this embodiment, the constraint between lane lines is not considered.

[0137] 5.1 Adding scatter factor The scatter factor can be a prediction point or a detection point. , it can be assigned to the corresponding lane segment. Specifically, if , then the scattered point is assigned to the nth lane line, and the residual from the scattered point to the corresponding lane line segment cubic polynomial curve can be calculated to construct the scatter factor:

[0138] ;

[0139] in, is the cubic curve coefficient corresponding to the nth lane line. Correspondingly, the variance of the scatter factor is the variance of the corresponding prediction point or detection point.

[0140] 5.2 Adding intra-segment constraint factors In this embodiment, the lane line is expected to be as flat as possible, so the intra-segment constraint factor makes the high-order terms of the cubic curve zero as much as possible, that is:

[0141] ;

[0142] In this embodiment, the covariance of the intra-segment constraint factor takes a fixed value of 0.0000001.

[0143] 5.3 Adding Inter-segment Constraint Factors In this embodiment, it is hoped that the two cubic polynomial curves transition continuously and smoothly, that is, it is hoped that the 0th, 1st, and 2nd order derivatives of the curves at the intersection point are continuous. Then:

[0144] = ;

[0145] in, is the cubic curve coefficient corresponding to the n-1th lane line, is the cubic curve coefficient corresponding to the nth lane line,

[0146] ;

[0147] is the inter-segment constraint coefficient matrix, The x value that separates the two segments. The covariance of the inter-segment constraint factor is fixed in this example (0.01, 0.001, 0.0001).

[0148] Step 6: A complete optimization problem can be constructed from all the above factors. The commonly used Gauss-Newton, Levenberg-Marquardt and other algorithms can be used to directly solve and obtain the cubic polynomial curve coefficients of each lane segment.

[0149] Step 7: Track update. Track update includes lane line sampling to generate update point set and variance update.

[0150] 7.1 Sampling Update Based on the cubic curve coefficient optimization results of each lane segment obtained in step 6, equidistant sampling is performed at fixed intervals in the x direction to obtain an updated point set for the lane line. Specifically, in this embodiment, sampling is performed at intervals of 1m. For the i-th sampling position of the n-th lane line, , have , is the corresponding sampling point. All lane segments of each lane line are sampled, and all sampling points are combined together to form the update point set of the corresponding lane line.

[0151] 7.2 Variance update. For each point in the update point set, traverse the prediction point set and the detection point set respectively to find the closest prediction point and detection point. Use Kalman filtering to calculate the variance of the update point:

[0152] ;

[0153] in, is the updated variance, is the variance of the nearest prediction point, is the variance of the nearest detection point. For unassociated tracks, ; For new tracks, .

[0154] The device for lane segmentation modeling and tracking processing based on the combination of filtering and optimization, wherein the device comprises:

[0155] a processor configured to execute computer executable instructions;

[0156] A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization are implemented.

[0157] The processor for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization are implemented.

[0158] The computer-readable storage medium stores a computer program thereon, and the computer program can be executed by a processor to implement the steps of the above-mentioned method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization.

[0159] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0160] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device.

[0161] A person of ordinary skill in the art may understand that all or part of the steps of the method for implementing the above-mentioned embodiment may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0162] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0163] In the description of this specification, the description with reference to the terms "an embodiment", "some embodiments", "example", "specific example", or "embodiment" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0164] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

[0165] The method, device, processor and computer-readable storage medium of lane line segmentation modeling and tracking processing based on the combination of filtering and optimization of the present invention are adopted. Lane line segmentation modeling can improve the disadvantage of insufficient characterization ability in single-segment curve modeling and improve the scene generalization ability; based on the hybrid strategy of filtering and optimization, historical frame information and its uncertainty can be transmitted in a recursive manner, while ensuring tracking stability, effectively avoiding the storage and maintenance of historical multi-frame data, improving the efficiency and real-time performance of the algorithm, and having certain practical value.

[0166] In this specification, the present invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and variations may be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method for lane segmentation modeling and tracking based on a combination of filtering and optimization, characterized in that: The method comprises the following steps: (1) Based on the lane line update point set of the previous frame and the odometer information of the current vehicle, the lane line update point set of the previous frame is transformed into the current frame ego vehicle coordinate system to obtain the prediction point set, thus completing the prediction processing for the lane line of the previous frame; (2) Associating the lane line prediction point set with the lane line detection point set of the current frame to construct an association cost matrix; (3) Based on the association results, the lane line trajectory is managed and the start and end points of the lane line are calculated to determine the range to be optimized; (4) Based on the start and end points of the lane line trajectory, lane line segmented cubic polynomial curve modeling is performed; (5) Using the factor graph to construct an optimization problem and solve it to obtain the modeling parameters of each lane segment; (6) Based on the optimization results, the update point set and corresponding variance of each lane line are calculated to complete the track update; The step (4) will segment the lane lines according to the starting and ending points of each lane line track, specifically: The starting and ending points are and ,according to x To preset interval s Divided into one section, then [ , ) is a segment, [ , ) is the next segment, and so on; for the last segment, the length does not meet the interval s of, that is ,in n is the current cumulative number of segments, then the last segment range is [ , ) ; Each lane line is represented by a cubic polynomial curve Modeling is performed, where , , , is the curve coefficient, i.e. the variable to be optimized, and y is the y-axis coordinate of any point on the curve; The step (5) specifically includes the following steps: (5.1) constructing scattered point constraints, lane segment constraints and lane segment constraints based on the prediction point set and the detection point set respectively; (5.2) Combining the above constraints, the factor graph is used to construct an optimization problem to solve, thereby obtaining the cubic polynomial curve coefficients of each lane segment; The step (5.1) described above constructs the scatter point constraints as follows: According to the coordinates of each scattered point , assign it to the corresponding lane segment. Specifically, if , then assign the scattered point to the n The residuals of the scattered points to the corresponding lane line segment cubic polynomial curve are calculated as follows to construct the scattered point factor : ； in, For the n The cubic curve coefficient corresponding to the lane line segment; The above step (5.1) constructs the lane segment constraints in the following way: ； in, is the intra-segment constraint factor; The step (5.1) described above constructs the lane segment constraints in the following way: = ； in, For the n -1 The cubic curve coefficient corresponding to the lane line, For the n The cubic curve coefficient corresponding to the lane line segment, ； is the inter-segment constraint coefficient matrix, Divided into two sections x value.

2. The method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization according to claim 1 is characterized in that: The step (1) is specifically as follows: Get the displacement vector of the lane lines in two adjacent frames based on the odometer information of the current vehicle And the corresponding corner , and construct the rotation transformation relationship as follows: ； in, Update the points in the point set for the previous frame, is the corresponding prediction point; All the points in the lane line update point set of the previous frame are transformed to the current coordinate system according to the above formula to obtain the predicted point set of the current frame.

3. The method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization according to claim 1 is characterized in that: The step (2) specifically includes the following steps: (2.1) For each lane line prediction point set, calculate the distance between it and each lane line detection point set, so as to construct the association cost matrix; (2.2) Use the Hungarian algorithm to find the matching result with the minimum total association cost.

4. The method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization according to claim 3 is characterized in that: The step (2.1) specifically includes the following steps: (2.1.1) First, obtain the prediction point set and the detection point set in x The intersection range of the two point sets in the direction is calculated. x The area enclosed by the intersection range is finally divided by the length of the intersection range to obtain the integrated average distance; (2.1.2) After calculating the integrated average distance between each prediction point set and each detection point set, fill it into the corresponding rows and columns of the associated cost matrix to complete the matrix construction.

5. The method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization according to claim 4 is characterized in that: The step (3) specifically includes the following steps: (3.1) For the unassociated detection point set, a new track id is assigned to it as a new lane line track. For the new lane line track, the starting and ending points of the detection point set are the starting and ending points of the lane line; (3.2) For the associated lane line trajectory, the starting endpoint of the lane line is selected from the closest point of the predicted point set and the closest point of the detection point set, and the ending endpoint of the lane line is selected from the farthest point of the predicted point set and the farthest point of the detection point set, whichever is farther. (3.3) For unassociated lane line trajectories, the starting and ending points of the predicted point set are the starting and ending points of the lane line.

6. The method for lane segmentation modeling and tracking processing based on the combination of filtering and optimization according to claim 1 is characterized in that: The step (6) specifically includes the following steps: (6.1) The cubic curve coefficient optimization results of each lane segment are obtained and x Perform equidistant sampling at fixed intervals in the direction, and combine all sampling points together to obtain the update point set of the corresponding lane line; (6.2) For the associated track, for each point in the update point set, traverse the corresponding prediction point set and detection point set respectively, find the nearest prediction point and detection point, and use Kalman filtering to calculate the variance of the update point: ； in, is the updated variance, is the variance of the nearest prediction point, is the variance of the nearest detection point; For unassociated tracks, ; For new tracks, .

7. A device for lane segmentation modeling and tracking processing based on a combination of filtering and optimization, characterized in that: The device comprises: a processor configured to execute computer-executable instructions; A memory storing one or more computer executable instructions, wherein when the computer executable instructions are executed by the processor, the steps of the method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization as described in any one of claims 1 to 6 are implemented.

8. A processor for lane segmentation modeling and tracking based on a combination of filtering and optimization, characterized in that: The processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method for lane line segmentation modeling and tracking processing based on the combination of filtering and optimization as described in any one of claims 1 to 6.

Images