A computer-implemented method for generating training data for training a machine learning model

Abstract

A computer-implemented method for generating training data for training a machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels, the method comprising: obtaining an image captured by the autonomous vehicle during travel along the path; projecting a lane boundary model from a 3D LiDAR point cloud of the path onto the image; and generating an example of training data for training the machine learning model by automatically removing one or more portions of the lane boundary corresponding to one or more respective occluded portions within the image.

Description

Technical Field

[0001] The subject matter of the present disclosure relates to a computer-implemented method for generating training data for training a machine learning model to automatically detect lane boundaries from an image of a path on which an autonomous vehicle travels, a computer-implemented method for training a machine learning model to automatically detect lane boundaries from an image of a path on which an autonomous vehicle travels, and a non-transitory computer-readable medium.

Background Art

[0002] An autonomous vehicle (AV) uses various sensors to guide a path. For example, the AV may navigate a path using an image captured by the AV. In order to navigate the path appropriately, it is necessary for the AV to understand where the lane boundaries are along the path.

[0003] By using a machine learning model, it can be useful for estimating lane boundaries using real-time images. However, training a machine learning model for this purpose currently requires training using manual annotation of a large number of training examples of past runs of the path. This is because it is difficult to automate the generation of lane boundaries considering other objects that obscure various parts of the lane boundaries.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The subject matter of the present disclosure aims to improve the prior art by alleviating such problems.

Means for Solving the Problems

[0005] According to one aspect of the present disclosure, there is provided a computer-implemented method for generating training data for training a machine learning model to automatically detect lane boundaries from an image of a path on which an autonomous vehicle travels, the method comprising: obtaining an image captured by the autonomous vehicle during travel on the path; projecting a lane boundary model from a three-dimensional LiDAR point cloud of the path onto the image; and generating a training data example for training the machine learning model by automatically removing one or more portions of the lane boundary corresponding to one or more occluded portions in the image.

[0006] By automatically removing one or more portions of the lane boundary, the time for training the machine learning model is significantly reduced, and the machine learning model can be extended to different regions and new paths.

[0007] Training a machine learning model to automatically detect lane boundaries from an image of a path on which an autonomous vehicle travels means automatically detecting a lane boundary without occlusions, a lane boundary without occlusions, or a lane boundary from which one or more occluded portions have been removed from an image of a path on which an autonomous vehicle travels. As used herein, the term "occluded portion" may be used to mean any portion of a lane boundary that is hidden or obscured by another feature on the path and thus not visible. Examples of such features include other vehicles, roadworks, puddles, and the like.

[0008] As used herein, the term "lane boundary" may be used to define an interface between an area where an AV can travel and an area where an AV cannot travel. The interface may be structural, such as a curb between a road and a sidewalk / pedestrian path (pavement). The interface may be non-structural, such as a lane dividing line on a road.

[0009] In one embodiment, the step of automatically removing one or more portions of the lane boundary may include generating a free space model including one or more free space regions and one or more occluded regions from the image using a further machine learning model, deleting one or more portions of the lane boundary that overlap with the one or more occluded regions, and retaining one or more portions of the lane boundary that overlap with the one or more free space regions.

[0010] In one embodiment, the computer-implemented method may further include generating ground truth including the one or more retained portions of the lane boundary.

[0011] In one embodiment, the further machine learning model may include a neural network.

[0012] In one embodiment, the neural network may be a convolutional neural network.

[0013] In one embodiment, the computer-implemented method may further include obtaining a new image captured by the autonomous vehicle during different runs of the route, and performing an estimation on the new image using the machine learning model to determine a new lane boundary.

[0014] The determined lane boundary may be an occlusionless lane boundary or a lane boundary without occlusions.

[0015] In one embodiment, the computer-implemented method may further include comparing the new lane boundary with the ground truth, and determining an accuracy score of the new lane boundary based on the comparison.

[0016] In one embodiment, determining the accuracy score may be determined as one or more of the precision score based on the comparison, and / or the recall score based on the comparison, and / or the f1 score based on the comparison.

[0017] In one embodiment, the computer-implemented method further includes determining that the machine learning model is accurate when the accuracy score is greater than or equal to an accuracy threshold, and determining that the machine learning model is not accurate when the accuracy score is less than the accuracy threshold.

[0018] In one embodiment, when it is determined that the machine learning model is not accurate, the computer-implemented method further includes repeating, using the new image as the image, projecting the lane boundary model from the 3D LiDAR point cloud of the path onto the image, and generating a training data example for training the machine learning model by automatically removing one or more portions of the lane boundary corresponding to one or more respective occluded portions in the image.

[0019] In one embodiment, the computer-implemented method further includes acquiring a plurality of LiDAR points of the path by the autonomous vehicle, and integrating the plurality of LiDAR points to generate a 3D point cloud of the path.

[0020] In one embodiment, the plurality of LiDAR points and the image are paired.

[0021] In one embodiment, the computer-implemented method further includes: identifying a lane boundary from the captured image using a machine learning algorithm; constructing a three-dimensional point cloud of the lane boundary by selecting a plurality of points that are positionally corresponding to the identified lane boundary from the integrated three-dimensional point cloud; clustering a plurality of points from the three-dimensional point cloud of the lane boundary into one or more clusters using an inter-point distance; and constructing an optimal spline for each cluster as the lane boundary model.

[0022] In one embodiment, the inter-point distance is calculated by determining a distance between each point and its respective adjacent point, and clustering the plurality of points into the cluster if their respective distances are less than a distance threshold.

[0023] In one embodiment, the distance threshold may be weighted according to a direction to its respective adjacent point.

[0024] In one embodiment, the distance threshold may be weighted to increase in a first direction and decrease in a second direction, the first direction being parallel to a direction of travel of the autonomous vehicle, and the second direction being perpendicular to the direction of travel of the autonomous vehicle.

[0025] In one embodiment, selecting a plurality of the points includes: repeatedly selecting a set of random points from the plurality of points within the cluster; constructing an optimal spline for each repeatedly selected set; calculating a distance from the optimal spline to the points within the set for each set; and selecting the optimal spline with the smallest distance as the optimal spline of the cluster.

[0026] In one embodiment, the distance may be a total distance.

[0027] In one embodiment, the distance may be an average distance.

[0028] In one embodiment, the machine learning model may include a neural network.

[0029] In one embodiment, the neural network may be a convolutional neural network.

[0030] According to one aspect of the present disclosure, a computer-implemented method for training a machine learning model to automatically detect lane boundaries from an image of a path on which an autonomous vehicle travels, the method comprising: providing a plurality of training data examples of images of a path on which an autonomous vehicle travels, each image including a lane boundary; providing, from the image, a lane boundary model without a plurality of occlusions corresponding to the lane boundary; and training a machine learning model to automatically detect a lane boundary without an occlusion from an image of a path on which an autonomous vehicle travels.

[0031] In one embodiment, the step of obtaining a plurality of the training data examples includes the above-described computer-implemented method.

[0032] According to one aspect of the present disclosure, a computer-implemented method for determining a trajectory along which an autonomous vehicle travels without crossing a detected lane boundary of a path, the method comprising: capturing an image of the path; using a machine learning model to estimate a lane boundary within the image; determining a trajectory along which the autonomous vehicle moves along the path without crossing the estimated lane boundary; and controlling the autonomous vehicle to move along the trajectory.

[0033] In one embodiment, the machine learning model is the machine learning model from the foregoing aspects and embodiments.

[0034] According to one aspect of the present disclosure, there is provided a temporary or non-temporary computer-readable medium storing instructions that, when executed by a processor, cause the processor to execute the method according to any one of the preceding claims.

[0035] According to one aspect of the present disclosure, there is provided an autonomous vehicle including a non-temporary computer-readable medium.

Brief Description of the Drawings

[0036] The subject matter of the present disclosure is best described with reference to the accompanying drawings.

[0037]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

[0038] The embodiments described herein are embodied as a set of instructions stored as electronic data on one or more storage media. Specifically, the instructions may be provided on a transient or non-transient computer-readable medium. When executed by a processor, the processor is configured to execute the various methods described in the following embodiments. Thus, these methods may be computer-implemented methods. In particular, the processor and the storage containing the instructions may be incorporated into a vehicle. The vehicle may be an AV.

[0039] The following embodiments provide specific exemplary examples, but those exemplary examples should not be construed as limiting, and the scope of protection is defined by the claims. The features of a particular embodiment may be used in combination with the features of other embodiments to the extent that they do not expand the subject matter beyond the content of the present disclosure.

[0040] As shown in FIG. 1, AV10 may include a plurality of sensors 12, 13. Sensor 12 may be attached to the roof of AV10. Sensor 13 may be attached to the front grille at the front of AV10. Sensors 12, 13 may be communicably connected to a computer 14. Computer 14 may be mounted on AV10. Computer 14 may include a processor 16 and a memory 18. The memory may include the non-transitory computer-readable medium described above. Alternatively, the non-transitory computer-readable medium may be located remotely and communicably linked to computer 14 via cloud 20. Computer 14 may be communicably linked to one or more actuators 22 to control the actuators 22 to move AV10. The actuators may include, for example, motors, braking systems, power steering systems, and the like.

[0041] Sensors 12, 13 may include various sensor types. Examples of sensor types include LiDAR sensors, RADAR sensors, and cameras. Each sensor type may be referred to as a sensor modality. Each sensor type may record data associated with the sensor modality. For example, a LiDAR sensor may record LiDAR modality data. When sensor 13 is attached to the front of AV10, in order to more easily detect features such as lane boundaries behind a specific shielding portion (e.g., a vehicle) when attached near the ground, sensor 13 may include a RADAR sensor.

[0042] The data may capture various scenes that AV10 encounters. For example, the scene may be the visible scene around AV10 and may include roads, buildings, weather, objects (such as other vehicles, pedestrians, animals, etc.).

[0043] As shown in FIG. 2, according to one or more embodiments, a computer-implemented method is provided for training a machine learning model to automatically detect lane boundaries from an image of a path along which an AV travels. As part of this method, a computer-implemented method for generating a lane boundary model may be provided.

[0044] To generate a 3D LiDAR point cloud of the path, the method may include, in step 100, obtaining a plurality of LiDAR points of the path along which AV 10 is moving. Step 100 may also include obtaining one image or a plurality of images of the same path. The image may be captured by a sensor 12 mounted on the roof in the form of one or more cameras. The LiDAR points may be captured by a sensor 13 in the form of a LiDAR sensor. The LiDAR sensor 13 mounted on the front of the AV 10 enables the LiDAR beam to pass under specific objects such as parked vehicles that may block the lane boundary. The LiDAR points and the image may be paired. In other words, the LiDAR points and the image captured by the AV 10 may be synchronized because they are linked to the same system clock of the computer 14.

[0045] In step 102, a machine learning model is provided. The machine learning model may include a neural network. The neural network may be a convolutional neural network. The machine learning model may be trained to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels. First, the machine learning model is trained using manually labeled training data of images having manual labels representing lane boundaries on the path within the image, as described below.

[0046] In step 104, the method includes identifying one or more lane boundaries within one or more images captured in step 100 using the machine learning model. The lane boundaries are stored as automatically generated labels as shown in 106.

[0047] As briefly shown in FIG. 3, a plurality of LiDAR points may be integrated to form a 3D LiDAR point cloud of path 200.

[0048] As further shown in FIG. 2, in step 108, the 3D LiDAR point cloud of path 200 is annotated at 108 to render an annotated map 110. As will be described below, the annotated map 110 includes a lane boundary model.

[0049] As shown in FIG. 4, a 3D LiDAR point cloud of lane boundary 202 may be constructed by selecting a plurality of points from the integrated 3D point cloud of path 200 that are positionally corresponding to the identified lane boundaries. Since the images and LiDAR used to derive the lane boundaries using a machine learning model are paired, the points of the 3D point cloud of the path that are positionally corresponding to the lane boundaries can be selected as the 3D LiDAR point cloud of lane boundary 202. FIG. 5 shows the same method of constructing the 3D point cloud of lane boundary 202 for different paths.

[0050] As shown in FIG. 6, a plurality of points from the 3D point cloud of the lane boundaries are clustered into clusters 202A - D for each specific lane boundary. Since there are four lane boundaries in FIG. 6, there are four clusters 202A - D. The step of clustering the points into clusters is performed using the distance between points. More specifically, the distance between points may be the distance between each point and its adjacent points. When the respective distances are less than a distance threshold, the plurality of points are clustered into clusters, and the distance threshold is weighted according to the direction to its adjacent points.

[0051] The distance threshold is weighted to increase in the first direction and decrease in the second direction. The first direction may be parallel to the moving direction of the autonomous vehicle, and the second direction may be perpendicular to the moving direction of the autonomous vehicle.

[0052] For example, select the first point. Identify the points adjacent to the first point. Measure the distance between the first point and each of the identified adjacent points. The distance can be derived from the position information of the LiDAR scan. The distance may be a real-world distance.

[0053] The distance threshold may be weighted to increase non-linearly. For example, in a plan view, the distance threshold may appear elliptical. For example, if the first adjacent point is X mm away from the first point in the longitudinal direction of the AV's movement and the second adjacent point is also X mm away from the first point in the transverse direction of the AV's movement, the first adjacent point may be within the boundary of the distance threshold, while the second adjacent point may not be within the boundary of the distance threshold.

[0054] Figure 7 shows a diagram similar to Figure 6 of the clusters obtained for the lane boundaries of different routes. There are five clusters 202A to E in Figure 7.

[0055] Next, as shown in Figure 8, use a spline generation algorithm to construct an optimal spline for each cluster. To construct an optimal spline, first, repeatedly select a set of random points from multiple points within the cluster. There may be 10 3 iterations. For example, there may be about 2000 iterations. In each iteration, a different set of points is randomly selected. Then, for each set, construct an optimal spline.

[0056] Next, calculate the distance from the optimal spline to the points within the set. The distance may be the total distance or the average distance. For example, the unit distance between each point within the set and the spline may be calculated. These unit distances may be the smallest distances between each point and the spline. In other words, the point on the spline that is closest to each respective point is used to measure the unit distance. The total distance is obtained by summing all the unit distances. The average distance is obtained by dividing the total distance by the number of points within the set. Select the optimal spline with the smallest distance as the optimal splines 204A - D of the cluster. Combine all the optimal splines 204A - D of the image to form a lane boundary model. FIG. 9 shows a similar view to FIG. 8, and the same method is used for different routes.

[0057] As further shown in FIG. 2, as described above, in 108, the lane boundary model annotates the 3D LiDAR point cloud (map) of the route to form an annotated map 110.

[0058] The method may also include a computer - implemented method for generating training data for training a machine - learning model to automatically detect lane boundaries from an image of a route along which an autonomous vehicle travels.

[0059] In step 112, based on the 3D LiDAR point cloud (map) of the route, project the lane boundary model onto a plurality of traversals of the route. In this specification, the term "traversal" is used to mean different journeys along the same route. The lane boundary model should be valid for all traversals, but certain portions of the lane boundary are occluded because an object appears along the route in some traversals but not in others. In particular, dynamic objects may vary between traversals.

[0060] Therefore, in step 114, one or more portions of one or more lane boundaries may be removed to delete the occluded portions of the lane boundary. Automatically remove one or more portions of the lane boundary.

[0061] As shown in FIG. 10, the automatic removal of one or more portions of the lane boundary may include the step of generating a free space model from the image using a further machine learning model. The free space model includes one or more free space regions and one or more occluded regions. The further machine learning model may include a neural network. The neural network may be a convolutional neural network.

[0062] The further machine learning model is initially trained using pairs of manually annotated free space boundaries and images. The boundary 208 separates one or more free space regions 210 and one or more occluded regions 212. Thus, the further machine learning model is trained to generate a free space boundary 208 between one or more free space regions 210 and one or more occluded regions 212.

[0063] As shown in FIGS. 2 and 10, the method may include, in step 114, the step of deleting one or more portions overlapping with one or more occluded regions of the lane boundary and the step of retaining one or more portions overlapping with one or more free space regions of the lane boundary. Thus, the method generates a lane boundary without occlusions including only the retained one or more portions of one or more lane boundaries in the form of the ground truth 116. Also, a training example 118 including an image annotated with a lane boundary without occlusions is generated. In step 120, the live image of the driving lane and the image annotated with a lane boundary without occlusions may be used as a training pair for training the machine learning model 102.

[0064] As further shown in FIG. 2, in step 122, as described above, the path image may be manually annotated initially to train the machine learning model 102.

[0065] In step 124, the method may include the step of capturing new images by the AV 10 during different runs of the path. Since point clouds are not required during estimation, it is not necessary to capture LiDAR data at this stage.

[0066] In step 126, the machine learning model 102 is used to perform an estimation on the new image to identify a new lane boundary. The new lane boundary is indicated by 128.

[0067] In step 130, the new lane boundary 128 is compared with the ground truth 116 to evaluate the performance of the machine learning model 102. Based on the comparison, an accuracy score of the new lane boundary may be determined. More specifically, the accuracy score may be determined by one or more of the following methods. In other words, the accuracy score may be equivalent to any of a precision score, a recall score, or an F1 score. Based on the comparison, a precision score may be determined. Based on the comparison, a recall score may be determined. Based on the comparison, an F1 score may be determined.

[0068] The aforementioned scores are calculated by different formulas using false negatives (FN), false positives (FP), true negatives (TN), and true positives (TP) of the results obtained by comparison. These different formulas may be as follows. Accuracy score = (TP + TN) / (TP + FP + FN + TN)

[0069] Put simply, the accuracy score is the ratio of correctly predicted observations to the total number of observations. Precision score = TP / (TP + FP)

[0070] Therefore, the precision score is the ratio of correctly predicted positive observations to the total number of predicted positive observations. Recall score = TP / (TP + FN)

[0071] Therefore, the recall score is the ratio of correctly predicted positive observations to all observations within the actual class. F1 score = 2 × (recall score × precision score) / (recall score + precision score)

[0072] Therefore, the F1 score is the weighted average of the precision score and the recall score.

[0073] In step 130, if the accuracy score is greater than or equal to the accuracy threshold, the machine learning model 102 is considered to have made an accurate prediction. In other words, the machine learning model is considered to be accurate. If the accuracy score is less than the accuracy threshold, the machine learning model 102 is considered to have not made an accurate prediction. In other words, the machine learning model is considered to be inaccurate.

[0074] When the machine learning model 102 is considered to have not made an accurate prediction, during further iterations of the method, step 112 is repeated using a new image as the above image, and the lane boundary model is projected onto the new image 124. Then, steps 114, 118, and 120 are executed to improve the accuracy of the machine learning model 102 for the acquired driving line of the new image. In 116, a new ground truth may be generated for the new image.

[0075] In summary, as shown in FIG. 11, a computer-implemented method for generating training data for training a machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels, the method including step S300 of acquiring an image captured by the autonomous vehicle during travel along the path, step S302 of projecting a lane boundary model from a three-dimensional LiDAR point cloud of the path onto the image, and step S304 of generating an example of training data for training the machine learning model by automatically removing one or more portions of the lane boundary corresponding to one or more occluded portions in the image, is provided.

[0076] Also, as shown in FIG. 12, a computer-implemented method for training a machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels, the method comprising: a step S400 of obtaining a plurality of training data examples of an image of a path driving line in which a lane boundary without a shielding portion is identified; and a step S402 of training a machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels. A computer-implemented method is provided.

[0077] The same method using step 124, step 126, step 128, and optionally step 130, step 132 can be used at runtime. In this way, the method of controlling a vehicle to travel along a path is as described above. In other words, as shown in FIG. 10 according to one or more embodiments, a computer-implemented method for determining a trajectory along which an autonomous vehicle travels without crossing a detected lane boundary of a path, the method comprising: a step S500 of capturing an image of the path; a step S502 of estimating a lane boundary in the image using a machine learning model; a step S504 of determining a trajectory along which the autonomous vehicle moves along the path without crossing the estimated lane boundary; and a step S506 of controlling the autonomous vehicle to move along the trajectory. The machine learning model may be the machine learning model 102 of FIG. 2.

[0078] The foregoing embodiments have been described for the purpose of explaining the subject matter of the present disclosure, but the features of the embodiments should not be construed as limiting the scope of protection. To avoid doubt, the scope of protection is defined by the following claims.

[0079] The following includes one or more items providing information related to the subject matter of the present disclosure.

[0080] Item Item 1. A computer-implemented method for generating a lane boundary model of a route on which an autonomous vehicle travels, the method comprising: obtaining a 3D LiDAR point cloud of the route and an image of the route on which the autonomous vehicle travels; detecting lane boundaries in the image of the route using a machine learning model; and generating a lane boundary model based on a plurality of points of the 3D LiDAR point cloud of the route that are positionally corresponding to the detected lane boundaries.

[0081] Item 2. The computer-implemented method according to Item 1, wherein the step of obtaining the 3D LiDAR point cloud of the route includes: capturing a plurality of LiDAR points of the route by the autonomous vehicle; and integrating the plurality of LiDAR points to generate a 3D LiDAR point cloud of the route.

[0082] Item 3. The computer-implemented method according to Item 2, wherein the plurality of LiDAR points and the image are paired.

[0083] Item 4. The computer-implemented method according to any one of Items 1 to 3, wherein the step of obtaining the image of the route includes: capturing the image of the route by the autonomous vehicle.

[0084] Item 5. The computer-implemented method according to any one of Items 1 to 4, wherein the step of generating a lane boundary model based on a plurality of points of the 3D LiDAR point cloud of the route that are positionally corresponding to the detected lane boundaries further includes: constructing a 3D point cloud of the lane boundary by selecting a plurality of points that are positionally corresponding to the identified lane boundary from the integrated 3D point cloud; clustering a plurality of points from the 3D point cloud of the lane boundary into one or more clusters using an inter-point distance; and constructing an optimal spline for each cluster as the lane boundary model.

[0085] Item 6. The computer-implemented method according to item 5, wherein the inter-point distance is calculated by determining the distance between each point and its respective adjacent point, and clustering the plurality of points into the cluster when each distance is less than a distance threshold.

[0086] Item 7. The computer-implemented method according to item 6, wherein the distance threshold is weighted according to the direction to the respective adjacent point.

[0087] Item 8. The computer-implemented method according to item 7, wherein the distance threshold is weighted so as to increase in a first direction and decrease in a second direction, the first direction being parallel to the moving direction of the autonomous vehicle, and the second direction being perpendicular to the moving direction of the autonomous vehicle.

[0088] Item 9. The step of constructing the optimal spline further includes repeatedly selecting a set of random points from the plurality of points within the cluster, constructing an optimal spline for each repeatedly selected set, calculating the distance from the optimal spline to the points within the set for each set, and selecting the optimal spline with the smallest distance as the optimal spline of the cluster, according to any one of items 5 to 8.

[0089] Item 10. The computer-implemented method according to item 9, wherein the distance is the total distance.

[0090] Item 11. The computer-implemented method according to item 9, wherein the distance is the average distance.

[0091] Item 12. The computer-implemented method according to any one of items 1 to 11, wherein the machine learning model includes a neural network.

[0092] Item 13. The computer-implemented method according to item 12, wherein the neural network is a convolutional neural network.

[0093] Item 14. A temporary or non - temporary computer - readable medium storing instructions that, when executed by a processor, cause the processor to execute the method according to any one of Items 1 to 13.

[0094] Item 15. An autonomous vehicle including the temporary computer - readable medium of Item 14.

Claims

1. A computer-implemented method for generating training data for training a machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels, comprising: obtaining an image captured by the autonomous vehicle during travel along the path; projecting a lane boundary model from a 3D LiDAR point cloud of the path onto the image; and generating a training data example for training the machine learning model by automatically removing one or more portions of the lane boundary corresponding to one or more occluded portions within the image. A computer-implemented method as described above.

2. The step of automatically removing one or more portions of the lane boundary comprises: generating a free space model including one or more free space regions and one or more occluded regions from the image using a further machine learning model; deleting one or more portions of the lane boundary that overlap with one or more of the occluded regions; and retaining one or more portions of the lane boundary that overlap with one or more of the free space regions. A computer-implemented method according to Claim 1.

3. A computer-implemented method according to Claim 2, further comprising generating a ground truth including the one or more retained portions of the lane boundary.

4. A computer-implemented method according to Claim 2 or 3, wherein the further machine learning model includes a neural network.

5. A computer-implemented method according to Claim 4, wherein the neural network is a convolutional neural network.

6. A computer-implemented method according to any one of Claims 1 to 5, further comprising: obtaining new images captured by the autonomous vehicle during different travels along the path; and performing an estimation on the new images using the machine learning model to determine new lane boundaries.

7. Comparing the new lane boundaries with the ground truth; and Determining an accuracy score for the new lane boundaries based on the comparison. A computer-implemented method according to Claim 6 when dependent on Claim 3.

8. Determining the accuracy score may be based on: a precision score based on the comparison, and / or a recall score based on the comparison, and / or an f1 score based on the comparison. A computer-implemented method according to Claim 7.

9. When the correct rate score is equal to or higher than the correct rate threshold, determining that the machine learning model is accurate; When the correct rate score is less than the correct rate threshold, determining that the machine learning model is not accurate; The computer-implemented method according to claim 7 or 8, further comprising:

10. When it is determined that the machine learning model is not accurate, Projecting the lane boundary model from the 3D LiDAR point cloud of the path onto the image; Generating a training data example for training the machine learning model by automatically removing one or more portions of the lane boundary corresponding to one or more respective occlusion portions in the image; The computer-implemented method according to claim 9, further comprising repeating, using the new image as the image.

11. Obtaining, by the autonomous vehicle, a plurality of LiDAR points of the path; Integrating the plurality of LiDAR points to generate a 3D point cloud of the path; The computer-implemented method according to any one of claims 1 to 10, further comprising:

12. The computer-implemented method according to claim 11, wherein the plurality of LiDAR points and the image are paired.

13. Identifying a lane boundary from the captured image using a machine learning algorithm; Constructing a 3D point cloud of the lane boundary by selecting a plurality of points corresponding in position to the identified lane boundary from the integrated 3D point cloud; Clustering a plurality of points from the 3D point cloud of the lane boundary into one or more clusters using an inter-point distance; Constructing an optimal spline for each cluster as the lane boundary model; The computer-implemented method according to claim 11 or 12, further comprising:

14. The inter-point distance is Determining a distance between each point and its respective adjacent point; When the respective distances are less than a distance threshold, clustering the plurality of points into the cluster; The computer-implemented method according to claim 13, wherein the inter-point distance is calculated by:

15. The computer-implemented method according to claim 14, wherein the distance threshold is weighted according to a direction to the respective adjacent points.

16. The distance threshold is weighted so as to increase in a first direction and decrease in a second direction, the first direction being parallel to the moving direction of the autonomous vehicle and the second direction being perpendicular to the moving direction of the autonomous vehicle, the computer-implemented method according to claim 15.

17. The step of constructing the optimal spline comprises: iteratively selecting a set of random points from the plurality of points within the cluster; constructing an optimal spline for each iteratively selected set; calculating, for each set, the distance from the optimal spline to the points within the set; selecting, as the optimal spline for the cluster, the optimal spline for which the distance is the smallest The computer-implemented method according to any one of claims 13 to 16.

18. The distance is the total distance, the computer-implemented method according to claim 17.

19. The distance is the average distance, the computer-implemented method according to claim 17.

20. The machine learning model includes a neural network, the computer-implemented method according to any one of claims 1 to 19.

21. The neural network is a convolutional neural network, the computer-implemented method according to claim 20.

22. A computer-implemented method of training a machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels, comprising: obtaining a plurality of training data examples of an image of a path travel line in which lane boundaries without occlusions are identified; training the machine learning model to automatically detect lane boundaries from an image of a path along which an autonomous vehicle travels The computer-implemented method.

23. The step of obtaining the plurality of training data examples includes the computer-implemented method according to any one of claims 1 to 20, the computer-implemented method according to claim 22.

24. A transient or non-transient computer-readable medium storing instructions that, when executed by a processor, cause the processor to execute the method according to any one of claims 1 to 23.

25. An autonomous vehicle comprising the non-transient computer-readable medium according to claim 24.

Images