Model-free lane tracking system

Abstract

A vehicle, a system, and a method for navigating the vehicle. The system includes a sensor and a processor. The sensor is configured to obtain a first set of detection points representing lanes of a road segment at a first time step, and obtain a second set of detection points representing lanes at a second time step. The processor is configured to determine a set of predicted points for a second time step from the first set of detection points, obtain a set of fused points from the second set of detection points and the set of predicted points, and navigate the vehicle using the set of fused points.

Description

Technical Field

[0001] The subject disclosure relates to a system and method for keeping a vehicle within a lane, and more particularly, to a method of tracking a lane including comparing a currently detected position of the lane to a predicted position of the lane without using a lane model. Background Art

[0002] A vehicle operating in an autonomous or semi-autonomous mode can maintain its position on a road segment by tracking the lanes or lane markings of the road segment over time. Some tracking methods use different modeling representations for lanes (e.g., polynomials, splines, etc.). Using lane modeling introduces several challenges, including model prediction and model updating, which are non-trivial and can be computationally expensive. In addition, using lane models can introduce fitting errors, especially in curved lanes and complex road geometries. Using a model reduces the data base that can lead to information loss. Therefore, it is desirable to provide a method for tracking lanes without using a lane model. Summary of the Invention

[0003] In one exemplary embodiment, a method for navigating a vehicle is disclosed. A first set of detection points representing lanes of a road segment is obtained at a sensor of the vehicle at a first time step. A set of predicted points at a second time step is determined at a processor from the first set of detection points. A second set of detection points representing lanes is obtained at the sensor at a second time step. A set of fused points is obtained at the processor from the second set of detection points and the set of predicted points. The vehicle is navigated at the processor using the fused set of points.

[0004] In addition to one or more features described herein, obtaining the set of fused points also includes forming a set of update points including the second set of detection points and the set of prediction points, forming a cluster of at least two update points, and forming a fused point for the cluster from the at least two update points within the cluster. Forming the fused point for the cluster also includes forming a weighted average of the at least two update points. The weighted average is based on uncertainty of the at least two update points. The method also includes filtering out the update points in the cluster using a criterion. The criterion is based on at least one of an uncertainty associated with the update point, an age of the update point, and a noise level of the update point. The detection points in the first set of detection points have a covariance matrix associated with the vehicle, and the method also includes determining a prediction point in the set of prediction points by updating the covariance matrix using a time interval between the first time step and the second time step.

[0005] In another exemplary embodiment, a system for navigating a vehicle is disclosed. The system includes a sensor and a processor. The sensor is configured to obtain a first set of detection points representing lanes of a road segment at a first time step and a second set of detection points representing the lanes at a second time step. The processor is configured to determine a set of predicted points for a second time step from the first set of detection points, obtain a set of fused points from the second set of detection points and the set of predicted points, and navigate the vehicle using the set of fused points.

[0006] In addition to one or more features described herein, the processor is further configured to obtain the set of fused points by forming a set of update points including the second set of detection points and the set of prediction points, forming a cluster of at least two update points, and forming a fused point of the cluster from the at least two update points within the cluster. The processor is further configured to form the fused point of the cluster by forming a weighted average of the at least two update points. The weighted average is based on uncertainty of the at least two update points. The processor is further configured to filter out the update points within the cluster using a criterion. The criterion is based on at least one of an uncertainty associated with the update point, an age of the update point, and a noise level of the update point. The detection points in the first set of detection points have a covariance matrix associated with the vehicle, and the method further includes determining the prediction point in the set of prediction points by updating the covariance matrix using a time interval between the first time step and the second time step.

[0007] In another exemplary embodiment, a vehicle is disclosed. The vehicle includes a sensor and a processor. The sensor is configured to obtain a first set of detection points representing lanes of a road segment at a first time step and a second set of detection points representing lanes at a second time step. The processor is configured to determine a set of predicted points for a second time step from the first set of detection points, obtain a set of fused points from the second set of detection points and the set of predicted points, and navigate the vehicle using the set of fused points.

[0008] In addition to one or more features described herein, the processor is further configured to obtain the set of fused points by forming a set of update points including the second set of detection points and the set of prediction points, forming a cluster of at least two update points, and forming a fused point of the cluster from the at least two update points within the cluster. The processor is further configured to form the fused point of the cluster by forming a weighted average of the at least two update points. The weighted average is based on uncertainty of the at least two update points. The processor is further configured to filter out the update points within the cluster using a criterion based on at least one of an uncertainty associated with the update point, an age of the update point, and a noise level of the update point.

[0009] The above features and advantages and other features and advantages of the present disclosure are readily apparent from the following detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Additional features, advantages and details appear by way of example only in the following detailed description, which refers to the accompanying drawings, in which:

[0011] Figure 1 is a plan view of a vehicle including a lane tracking system for vehicle navigation in an illustrative embodiment;

[0012] Figure 2 A block diagram of a method for tracking lanes or lane lines executed by a controller in one embodiment is shown;

[0013] Figure 3 Shown by Figure 2 The block diagram illustrates the timing diagram of the data flow;

[0014] Figure 4 A plan view of a road segment is shown, including a set of illustrative prediction points for a selected time step;

[0015] Figure 5 Shows a plan view of the road section, including Figure 4 The predicted points in and the second set of detection points obtained at the second time step;

[0016] Figure 6 shows a plan view of a road segment illustrating updated points generated by merging the predicted points with a second set of detected points;

[0017] Figure 7 shows a plan view of a road segment illustrating the operations occurring in the fusion module of the lane tracking system;

[0018] Figure 8 An illustrative example of the clustering process performed in the fusion module is shown;

[0019] Figure 9 shows the filtering steps performed in the fusion module; and

[0020] Figure 10 A schematic diagram illustrating point fusion is shown. DETAILED DESCRIPTION

[0021] The following description is merely exemplary in nature and is not intended to limit the present disclosure, its application or use. It should be understood that throughout the drawings, corresponding reference numerals indicate identical or corresponding components and features. As used herein, the term module refers to a processing circuit, which may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or grouped) and memory that executes one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the functionality described.

[0022] According to an exemplary embodiment, Figure 1FIG. 1 is a plan view of a vehicle 100 including a lane tracking system for vehicle navigation. Figure 1 The exemplary vehicle 100 in FIG. 1 is a car 101. Vehicle 100 may be an autonomous vehicle, a semi-autonomous vehicle, or may be manually driven by a driver. Vehicle 100 is shown within two lane markings 105 a, 105 b (generally represented here by 105 ) of road segment 102. The vehicle includes a controller 110 and one or more sensors 120 . Figure 1 The exemplary sensors 120 shown include one or more cameras 120a, a lidar system 120b, and a radar system 120c (collectively 120). The controller 110 can use one or more cameras 120a, lidar system 120b, or a combination thereof to obtain a detection point 104 for a given lane line (e.g., lane line 105a). Figure 1 The exemplary types, number, and locations of the sensors 120 shown in FIG are not intended to limit the types, number, and locations of the sensors 120 in any alternative embodiment of the vehicle 100. The sensors 120 acquire images of the detection points 104 of the lane lines at periodic time steps. In one embodiment, the sensors 120 acquire a set of detection points at a rate of approximately 100 times per second.

[0023] The controller 110 performs lane tracking using the methods discussed herein. The controller 110 can also implement semi-autonomous or autonomous control of the vehicle 100 based in part on lane tracking. The controller 110 obtains detection points 104 from raw images obtained by one or more sensors 120, performs lane tracking based on the detection points 104, and navigates the vehicle based on the lane tracking. For example, the controller 110 can perform image processing on one or more images obtained within the field of view of one or more cameras 120a surrounding the vehicle 100. The image processing can identify detection points 104 associated with one or more lane lines 105 (e.g., lane line 105a). Alternatively, a point cloud obtained using a lidar system 120b can be processed to obtain the detection points 104. To perform semi-autonomous or autonomous control of the vehicle 100, the lane lines 105a near the vehicle 100 are tracked based on the detection points 104. The methods disclosed herein represent lanes as sets of points without fitting these points to a model, such as a polynomial or spline. Bypassing channel modeling simplifies the prediction and update steps while maintaining a high degree of data cardinality. The method disclosed in this paper can accurately and efficiently track highly curved lanes in complex scenes.

[0024] Figure 2A block diagram 200 of a method for tracking lanes or lane lines performed by the controller 110 in one embodiment is shown. The method includes obtaining detection points in a first time step and predicting the positions of these detection points relative to the vehicle in a second time step. The predicted positions are compared with the detection points obtained by the sensor in the second time step. The method also includes an association step for finding the best overlap (overlay) of the predicted positions and the detection points in the second time step, and an update step for merging the points with the best overlap into a set of updated points. The method also includes a fusion step in which the updated points are grouped into clusters and representative points of the clusters are selected, which representative points are also referred to as fusion points in this document. The fusion step includes filtering or removing points from the clusters according to one or more selected criteria before determining the representative points.

[0025] The block diagram 200 includes various modules for executing the method disclosed herein, including a correlation module 202, an update module 204, a prediction module 206, and a fusion module 208. The method may loop through the correlation module 202, the update module 204, and the prediction module 206 at each of a plurality of time steps. The correlation module 202 receives a set of predicted points from the prediction module 206 and a set of detected points from the sensor 120. The predicted points are based on the detected points from the previous time step and are determined by calculation using one or more dynamic parameters of the vehicle.

[0026] Update module 204 receives the associated points and merges them into a set of updated points, where there is no distinction between predicted points and detected points. The updated points are considered to be detected points for the next cycle through block diagram 200. Prediction module 206 receives the updated points and predicts a set of predicted points for the next time step. Fusion module 208 creates a set of fused points from the updated points. This set of fused points provides representative points for the lane or lane line, which can be used by controller 110 for lane tracking and navigation.

[0027] The association module 202 performs an association step, which includes solving an assignment problem between a set of currently detected lanes and the set of predicted lanes, each representing a set of points. Solving the assignment problem includes evaluating the distance between each of the set of currently detected lanes and the set of predicted lanes and selecting the best lane-to-lane association. When assigning a point in the set of predicted lanes to a point in the set of currently detected lanes, an association cost is generated for the overlap. The association cost for the overlap is the sum of the association costs for each individual assignment. The set of detected lanes is then used to update the set of currently predicted lanes to which it is associated.

[0028] At update module 204, the detected set of lanes is used to update the set of predicted lanes by merging the two sets into a unified set of points, referred to as the updated set of points. This updating technique is simple and allows the detected points to be preserved without loss of information. To avoid the significant memory consumption caused by this updating method, points are filtered as part of the subsequent fusion method.

[0029] Referring to the prediction module 206, the points received at the prediction module include various measurement parameters, such as the coordinates (x, y) of the detection point, and dynamic parameters of the vehicle, such as the vehicle's speed, heading, acceleration, yaw rotation, etc. Assuming that the lane lines are static, a set of current predicted lanes at the current time step t can be obtained from the points from the previous time step by transforming the previous set of points at time (t-Δt) by inverse transforming the parameters about the vehicle handling measurement performed at t-Δt. The measured parameters can also include various uncertainties for each point. Each point can have a covariance matrix based on its parameters. Predicting a point at a subsequent time step includes updating the covariance matrix. In one embodiment, the covariance matrix Q(t) of the current time step can be approximated by the covariance matrix Q(t-Δt) of the previous time step, as shown in equation (1):

[0030]

[0031] Where x is the x-coordinate of the detection, y is the y-coordinate of the detection, and ω ego is the yaw rotation of the vehicle, is the uncertainty associated with the yaw rotation, is the uncertainty associated with the vehicle speed, and is the uncertainty associated with vehicle acceleration. In one embodiment, ΔQ can be approximated as:

[0032] Figure 3 Shown by the description Figure 2 A time diagram 300 of the data flow of the block diagram 200 is shown. For illustrative purposes, the time diagram includes only three time steps, although this is not meant to limit the number of time steps that may be executed by the controller 110.

[0033] An image or other sensory data is obtained from the sensor 120 at each time step. The image includes a set of detection points indicating lane lines. A first set of detection points obtained during a first time step (t) is denoted by O(t) (302), a second set of detection points obtained during a second time step (t+Δt) is denoted by O(t+Δt) (304), and a third set of detection points obtained during a third time step (t+2Δt) is denoted by O(t+2Δt) (306), where Δt is the time interval between time steps. The first set of detection points is used to generate a first set of predicted points P(t+Δt) (308) via the prediction module 206. The second set of detection points O(t+Δt) 304 is associated with the first set of predicted points P(t+1) via the association module 202 (308). The associated points are merged via the update module 204 to form a first set of updated points U(t+Δt) 310. The first set of updated points U(t+Δt) 310 is used to generate a first fused data set F(t+Δt) 312 via the fusion module 208. The first set of updated points U(t+Δt) 310 is also used to generate a second set of predicted points P(t+2Δt) 314 via the prediction module 206 at a third time step t+2Δt.

[0034] The process occurring during the third time step is similar to that of the second time step and includes associating the second set of predicted points P(t+2Δt) 314 to the third set of detected points O(t+2Δt) 306, forming a second set of updated points U(t+2Δt) 316, generating a second set of fused points F(t+2Δt) 318 from the updated points U(t+2Δt) 316, and generating a third set of predicted points for a subsequent time step.

[0035] Figure 4 A plan view 400 of the road segment 102 is shown, including an illustrative set of predicted points 402 for a selected time step (ie, the second time step (t+Δt)). Figure 3 As discussed in relation to the time diagram 300 of FIG. 1 , a predicted point 402 is generated from a detection point obtained during a first time step (t) when the vehicle 100 is at a first position 404. The predicted point 402 is a prediction of the detection point's position relative to the vehicle 100 at a second position 406 during a second time step (t+Δt).

[0036] Figure 5 A plan view 500 of a road segment 102 is shown, including Figure 4 and a second set of detected points 502 obtained at the second time step. Figure 6 A plan view 600 of the road segment 102 is shown illustrating updated points 602 generated by merging the predicted points 402 with the second set of detected points 502 .

[0037] Figure 7A plan view 700 of road segment 102 is shown, illustrating the operations occurring in fusion module 208. Fusion module 208 performs a clustering operation on updated points 602, as shown by first cluster 702a, second cluster 702b, third cluster 702c, and fourth cluster 702d. Within each cluster, a filtering process may be performed to remove points 604 that have lost their relevance to determining a fused point due to a selected reason. The updated points 602 remaining within a cluster are used to determine the representative point, or fused point, for that cluster. First fused point 704a represents first cluster 702a, second fused point 704b represents second cluster 702b, third fused point 704c represents third cluster 702c, and fourth fused point 704d represents fourth cluster 702d. The fused points are published to controller 110, which performs an action or maneuver on vehicle 100. The fusion algorithm used influences the points published to controller 110. The lane tracking system saves the original point set information for later use.

[0038] Figure 8 An illustrative example of the clustering process performed in the fusion module 208 is shown. A principal component analysis may be performed on the updated points to determine a set of feature vectors, such as x pca 802 and y pca 804. It is possible to follow the direction of the first eigenvector (i.e., x pca 802) The sample region is marked, as shown by marker 806. Update points within a selected proximity r of marker 806 may be grouped within a cluster 808 associated with the marker.

[0039] Figure 9 A filtering step 900 is shown as being performed at the fusion module 208. The filtering step is used to select updated points within the cluster 808 as candidates for the subsequent fusion step. Points (e.g., filter points 902) can be removed or filtered out based on selected criteria or quality metrics. For example, a point may have an associated uncertainty greater than a selected threshold, or may have an age greater than a selected time threshold and therefore have low relevance with respect to future calculations, etc. The noise level of the point may also be used as a filtering criterion. The step of removing points reduces the amount of memory consumed.

[0040] Figure 10 A diagram 1000 is shown of point fusion in an illustrative embodiment. Although shown with respect to two updated points, the fusion step may include fusion of any number of points. Fusion of the points may include obtaining a weighted average of at least two updated points. Figure 10In the illustrative example of , a first update point 1002 and a second update point 1004, both within the same cluster, are fused to form a fused point 1006. The fusion step assumes that each point is an uncorrelated Gaussian measurement of the same location in space. The fusion step for multiple points (e.g., point 1002 and point 1004) is mathematically shown in equation (3):

[0041]

[0042] in is the uncertainty of the first update point 1002, is the uncertainty of the second update point 1004. The values ​​T1 and T2 represent the parameters of the first update point 1002 and the second update point 1004, respectively. The uncertainty of the fusion point is given by equation (4):

[0043]

[0044] In various embodiments, the optimal parameters may be determined individually for each parameter (ie, x-coordinate, y-coordinate, heading, etc.), assuming there is no correlation between the uncertainties of these parameters.

[0045] Although the above disclosure has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope thereof. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the basic scope of the present disclosure. Therefore, the present disclosure is not limited to the particular embodiments disclosed, but is intended to encompass all embodiments falling within its scope.

Claims

1. A method for navigating a vehicle, comprising: Obtaining, at a first time step, a first set of detection points representing lanes of a road segment at a sensor of the vehicle; determining, at the processor, a set of predicted points for a second time step from the first set of detected points, the determination of the set of predicted points taking into account dynamic parameters of the vehicle, including speed, heading, acceleration, and yaw rotation of the vehicle; Obtaining a second set of detection points representing the lane at a second time step at the sensor; At the processor, obtaining a set of fused points from the second set of detection points and the set of predicted points, obtaining the set of fused points further comprising forming a set of update points including the second set of detection points and the set of predicted points, forming a cluster of at least two update points, and forming a fused point of the cluster from the at least two update points within the cluster; and At the processor, the set of fused points is used to navigate the vehicle.

2. The method according to claim 1, wherein Forming the clustered fusion points further includes forming a weighted average based on uncertainties of the at least two updated points.

3. The method of claim 1 , further comprising filtering out updated points within the cluster using criteria based on at least one of: (i) uncertainty associated with the updated point; (ii) age of the updated point; and (iii) noise level of the updated point.

4. The method according to claim 1, wherein A detection point in the first set of detection points has a covariance matrix associated with the vehicle, and further includes determining a predicted point in the set of predicted points by updating the covariance matrix using a time interval between a first time step and a second time step.

5. A system for navigating a vehicle, comprising: a sensor configured to obtain a first set of detection points representing lanes of a road segment at a first time step and to obtain a second set of detection points representing lanes at a second time step; and A processor configured to: determining a set of predicted points for a second time step from the first set of detected points, wherein the determination of the set of predicted points takes into account dynamic parameters of the vehicle, including speed, heading, acceleration, and yaw rotation of the vehicle; A set of fusion points is obtained from the second set of detection points and the set of prediction points, where The processor is further configured to obtain the set of fused points by forming a set of update points including the second set of detection points and the set of prediction points, forming a cluster of at least two update points, and forming a fused point of the cluster from at least two update points within the cluster; and The vehicle is navigated using the set of fused points.

6. The system according to claim 5, wherein: The processor is further configured to form a fusion point of the cluster by forming a weighted average based on uncertainties of the at least two update points.

7. The system according to claim 5, wherein: The processor is further configured to filter out update points within the cluster using criteria based on at least one of: (i) uncertainty associated with the update point; (ii) age of the update point; and (iii) the noise level at the update point.

8. The system according to claim 5, wherein: A detection point in the first set of detection points has a covariance matrix associated with the vehicle, and further includes determining a prediction point in the set of prediction points by updating the covariance matrix using a time interval between the first time step and the second time step.

Images