Method and system for lane tracking in an autonomous vehicle

Abstract

The disclosed subject matter relates to the field of autonomous vehicles, which discloses a method and system for training and lane tracking of a lane tracking system of an autonomous vehicle. During the training phase, the lane tracking system receives ground truth values ​​and measurements corresponding to lane boundary detection points, and determines ground truth clothoid points and measured clothoid points. Then, coefficient values ​​of clothoid parameters are determined for the measured clothoid points to model the lane boundaries, and then Kalman filter parameters for the coefficient values ​​are determined using at least one neural network to track the lane boundaries. Furthermore, the coefficient values ​​of the clothoid parameters are updated using the Kalman filter parameters, which are used to reconstruct the measured clothoid points and track the lane boundaries. Furthermore, the training error is obtained by the L2 norm between the clothoid points, rather than the coefficients, thereby enhancing the minimization of the training error. The trained lane tracking system is then deployed for lane tracking in a dynamic environment.

Description

[Technical Field]

[0001] The present subject matter relates generally to the field of autonomous vehicles, and more particularly, but not exclusively, to methods and systems for lane tracking in autonomous vehicles. [Background technology]

[0002] Currently, the automotive industry is beginning to move towards autonomous vehicles. As used herein and in the claims, an autonomous vehicle is a vehicle that can sense its surrounding environment to move on a road with or without human intervention. Autonomous vehicles sense the environment using sensors configured on the autonomous vehicle, such as lasers, light detection and ranging (LIDAR), global positioning systems (GPS), and computer vision. Autonomous vehicles heavily rely on lane detection and tracking on the road to move smoothly.

[0003] Existing lane detection and tracking techniques may use a Kalman filter to track lane boundaries. In particular, a Kalman filter may be used to predict lane parameters and smooth the output of a lane tracking device that tracks lane boundaries. Generally, a Kalman filter is chosen for tracking lane boundaries because it can estimate the dynamics of a state vector even in the presence of noisy measurements or noisy processes. Key parameters that help determine the Kalman filter are the process noise covariance matrix (Q) and the measurement noise covariance matrix (R). Existing lane detection and tracking techniques that rely on a Kalman filter to track lane boundaries use predefined or fixed values ​​of Q and R to determine the Kalman filter. In reality, Q and R are parameters that dynamically change based on the scenario, the detector used for the measurements, the type of process used for the measurements and tracking, etc. However, existing techniques fail to capture the dynamic nature of Q and R and instead use fixed or predefined values ​​for Q and R, which affects the accuracy of predictions made based on a Kalman filter for lane tracking. Inaccurate lane tracking can generate erroneous steering commands and warning signals to an autonomous vehicle, thereby jeopardizing the safety of the vehicle.

[0004] Additionally, existing techniques have fixed values ​​for Q and R, which makes them inflexible to incorporate changes in conditions over time, limiting predictions to only a few types of lane structures or a narrow range of lane structures. Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, there is a need for a method that provides greater accuracy and flexibility in lane tracking using Kalman filters.

[0006] The information disclosed in the Background section of this disclosure is intended only to enhance understanding of the general background of the disclosure, and should not be construed as an admission or any form of suggestion that this information constitutes prior art already known to those skilled in the art. [Means for solving the problem]

[0007] A method for training a lane tracking system for an autonomous vehicle is disclosed herein. The method includes receiving, by the lane tracking system, ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system associated with the lane tracking system. The method further includes determining coefficient values ​​of clothoid parameters for ground truth clothoid points and measured clothoid points formed using the ground truth set and the measured set, respectively, to model lane boundaries of the lane. The ground truth set includes a subset of consecutive lane boundary detection points and corresponding ground truth values, and the measured set includes a subset of consecutive lane boundary detection points and corresponding measurements. The method then includes determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track lane boundaries of the lane. The Kalman filter parameters are determined using at least one neural network. Upon determining the Kalman filter parameters, the method includes updating coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters. The method further includes reconstructing the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. Each reconstructed measured clothoid point enables the lane tracking system to track lane boundaries of the autonomous vehicle. Finally, the method includes minimizing, in each cycle, a training error based on differences between the reconstructed measured clothoid points and the corresponding ground truth set until the training error is below a predefined threshold.

[0008] The present disclosure further includes a lane tracking system for an autonomous vehicle. The lane tracking system includes a processor and a memory communicatively coupled to the processor. The memory stores processor instructions that, when executed, cause the processor to train the lane tracking system, where the processor is configured to receive, for training, ground truth values ​​corresponding to a plurality of lane boundary detection points and measured values ​​corresponding to the plurality of lane boundary detection points from a lane boundary detection system associated with the lane tracking system. The processor further determines coefficient values ​​of clothoid parameters for ground truth clothoid points and measured clothoid points formed using the ground truth set and the measured set, respectively, to model lane boundaries of the lane. The ground truth set includes a subset of consecutive lane boundary detection points and corresponding ground truth values, and the measured set includes a subset of consecutive lane boundary detection points and corresponding measured values. The processor then determines Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane. The Kalman filter parameters are determined using at least one neural network. After determining the Kalman filter parameters, the processor updates coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters. Furthermore, the processor reconstructs the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. Each reconstructed measured clothoid point enables the lane tracking system to track lane boundaries of the autonomous vehicle. Finally, in each cycle, the processor minimizes a training error based on the difference between the reconstructed measured clothoid points and the corresponding ground truth set until the training error is below a predefined threshold.

[0009] The present disclosure further discloses a method for lane tracking for an autonomous vehicle. The method includes receiving, by a lane tracking system, measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system associated with the lane tracking system. The method then includes determining coefficient values ​​of clothoid parameters for measured clothoid points formed using the measurement set to model lane boundaries of the lane. The measurement set includes a subset of consecutive lane boundary detection points and the corresponding measurements. The method then includes determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track lane boundaries of the lane. The Kalman filter parameters are determined using at least one neural network. Upon determining the Kalman filter, the method includes updating the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters. Finally, the method includes reconstructing the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. Each reconstructed measured clothoid point allows the lane tracking system to track the lane boundaries of the autonomous vehicle.

[0010] The present disclosure further discloses a lane tracking system for an autonomous vehicle. The lane tracking system includes a processor and a memory communicatively coupled to the processor. The memory stores processor instructions that, when executed, cause the processor to receive measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system associated with the lane tracking system. The processor then determines coefficient values ​​of clothoid parameters for measured clothoid points formed using the measurement set to model lane boundaries of the lane. The measurement set includes a subset of consecutive lane boundary detection points and the corresponding measurements. The processor then determines Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane. The Kalman filter parameters are determined using at least one neural network. Upon determining the Kalman filter, the processor updates the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters. Finally, the processor reconstructs the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. Each reconstructed measured clothoid point allows the lane tracking system to track the lane boundaries of the autonomous vehicle.

[0011] The above summary is illustrative only and is not intended to be in any way limiting. In addition to the exemplary aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

[0012] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate several exemplary embodiments and, together with the following description, serve to explain the disclosed principles. In the drawings, the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to refer to like features and elements. Some embodiments of systems and / or methods according to embodiments of the present subject matter will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0013] [Figure 1] 1 illustrates an example architecture for lane tracking for an autonomous vehicle, according to some embodiments of the present disclosure. [Figure 2A] FIG. 1 illustrates a detailed block diagram of an exemplary lane tracking system for lane tracking in an autonomous vehicle, according to some embodiments of the present disclosure. [Figure 2B] 1 illustrates an exemplary plurality of lane boundary detection points arranged consecutively according to exemplary ground truth values, in accordance with some embodiments of the present disclosure. [Figure 2C] 1 illustrates an exemplary plurality of lane boundary detection points arranged consecutively according to exemplary measurements, in accordance with some embodiments of the present disclosure. [Figure 2D] 1 illustrates an example lane tracked using example reconstructed clothoid points, in accordance with some embodiments of the present disclosure. [Figure 2E] 1 illustrates a number of exemplary tracked lanes belonging to a road on which an autonomous vehicle is traveling, according to some embodiments of the present disclosure. [Figure 3A] 1 shows a flowchart illustrating a method for training a lane tracking system of an autonomous vehicle, according to some embodiments of the present disclosure. [Figure 3B] 1 shows a flowchart illustrating a method for a lane tracking system for an autonomous vehicle, according to some embodiments of the present disclosure. [Figure 4] FIG. 1 is a block diagram of an exemplary computer system for implementing embodiments consistent with the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] It should be understood by those skilled in the art that the block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it should be understood that flowcharts, flow diagrams, state transition diagrams, pseudocode, and the like are substantially represented on a computer-readable medium and represent various processes that may be executed by such a computer or processor, whether or not a computer or processor is explicitly shown.

[0015] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or implementation of the subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0016] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and are described in detail below. It is to be understood, however, that it is not intended to limit the disclosure to the disclosed forms, but on the contrary, the disclosure is intended to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0017] The terms "comprise," "including," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a setup, device, or method that includes a list of components or steps does not include only those components or steps, but may also include other components or steps that are not expressly listed or inherent in such setup, or device, or method. In other words, one or more elements of a system or apparatus preceded by "comprise" does not exclude other or additional elements from being present in that system or method, unless there are further constraints.

[0018] Disclosed herein are methods and systems for lane tracking for autonomous vehicles. In some embodiments, lane tracking includes detecting the presence of one or more lanes on a roadway along which the autonomous vehicle is traveling and tracking the detected lanes to generate one or more commands for the autonomous vehicle. By way of example, the one or more commands may be a steering command, a braking command, a lane change command, an overtaking command, a warning signal, or the like, that facilitates movement of the autonomous vehicle. The present disclosure provides an artificial intelligence (AI)-based method for lane tracking for autonomous vehicles that addresses one or more problems of existing techniques described in the Background section of this disclosure. Because the method is an AI-based method, the lane tracking system disclosed in the present disclosure for tracking lanes of an autonomous vehicle requires training before deployment in a dynamic environment.

[0019] In some embodiments, the lane tracking system may be trained using ground truth values ​​corresponding to multiple lane boundary detection points and different types of lanes, along with measurements corresponding to multiple lane boundary detection points determined using image frames corresponding to various road scenarios. In some embodiments, ground truth values ​​may refer to original values, i.e., information known to be real and provided based on direct observation. However, measurements are values ​​determined or predicted by the system and are not based on direct observation like ground truth values. Thus, the measurements and ground truth values ​​may be the same or different, depending on the accuracy of the measurements. In some embodiments, a subset of the measurements may be used to form measured clothoid points, and a subset of the ground truth values ​​may be used to form ground truth clothoid points. During the training phase, the lane tracking system may be trained to determine a Kalman filter to update coefficient values ​​of clothoid parameters for the measured clothoid points and then reconstruct the measured clothoid points using the updated coefficient values ​​of the clothoid parameters. The lane tracking system may then determine a training error and be trained to minimize the training error per cycle. The training phase of the lane tracking system for tracking the lane of an autonomous vehicle based on clothoid points will be explained in detail with appropriate figures later in the detailed description of this disclosure.

[0020] During the training phase, the lane tracking system may determine Kalman filter parameters using one or more neural networks. The one or more neural networks may include a neural network with temporal memory, such as a recurrent neural network (RNN) or a long-short-term memory (LSTM) neural network. The one or more neural networks may store memories related to past events and may learn long-term dependencies based on the stored memories. In the present disclosure, during the training phase, the lane tracking system may be trained to dynamically determine the measurement noise covariance matrix (R) and the process noise covariance matrix (Q) using one or more of the same or different neural networks. For example, the measurement noise covariance matrix (R) may be dynamically determined using a first neural network, and the process noise covariance matrix (Q) may be dynamically determined using a second neural network. Each of the first neural network and the second neural network may be a neural network with temporal memory, such as an RNN or an LSTM network. In some other embodiments, the measurement noise covariance matrix (R) and the process noise covariance matrix (Q) may be dynamically determined using the same neural network. In reality, Q and R change dynamically based on the scenario, the detector used to measure, the type of process used to measure and track, etc. Using a neural network to determine Q and R allows the neural network to determine Q and R based on historical data captured from past cycles of the autonomous vehicle. Thus, Q and R may evolve together because the neural network determines Q and R by analyzing changes over time. Thus, the values ​​of Q and R determined using the neural network(s) are specific to the current scenario captured in the image frame, rather than some random, predefined or static value.Because the values ​​of Q and R are determined using a neural network that is data-driven, i.e., that analyzes based on historical data captured from past cycles and uses the analysis results to determine Q and R for the current scenario, the determined values ​​of Q and R are accurate and robust. Furthermore, because the neural network is trained using ground truth values ​​along with measurements during a training phase, the determined values ​​of Q and R are close to the ground truth values, thereby increasing the level of accuracy of the dynamically determined values ​​of Q and R. Such dynamically determined accurate values ​​of Q and R allow for the determination of an accurate Kalman filter for lane tracking, resulting in accurate updated coefficient values ​​for the clothoid parameters and reconstructed measured clothoid points.

[0021] In each cycle, a training error that may occur due to the reconstructed clothoid point may be determined based on the difference between the reconstructed clothoid point and a corresponding ground truth set, i.e., a subset of the ground truth values ​​of the corresponding lane boundary detection points. Such a training error may be minimized until the training error is below a predefined threshold. Therefore, by minimizing the training error during the training phase, even small training errors that may adversely affect the accuracy of lane tracking are reduced. In this disclosure, the training error may be obtained by using the L2 norm between the clothoid points, rather than using the clothoid coefficients used in some existing techniques. Therefore, in this disclosure, a new method for minimizing the error is continued, which includes comparing the reconstructed measured clothoid point with the ground truth values ​​of the corresponding lane boundary detection points that were originally used to form the reconstructed measured clothoid point. This enhances error minimization in a better way and with fewer cycles compared to conventional error minimization techniques that involve comparing updated coefficient values ​​of the clothoid parameters of measured clothoid points with coefficient values ​​of the clothoid parameters of ground truth clothoid points.

[0022] Additionally, because Q and R inherently represent process noise and measurement noise, respectively, determining Q and R requires data from one or more sensors configured in the autonomous vehicle. The use of a neural network, such as an RNN or LSTM network, may provide the present disclosure with the flexibility to include a sensor error model in the neural network. Such a sensor error model may provide low-level features associated with one or more sensors, such as the amount of noise associated with the measurement or the amount of noise associated with the measurement process. Such low-level features may help directly correct the sensor errors using the sensor error model, thereby improving the accuracy of the dynamically determined values ​​of Q and R. This, in turn, helps maximize lane tracking accuracy. Therefore, using a neural network and a Kalman filter to perform lane tracking based on clothoid parameters not only enables accurate lane tracking but also reduces the generation of erroneous steering commands and warning signals, thereby enhancing the safety of the autonomous vehicle.

[0023] A description of an embodiment having several components in communication with each other does not imply that all such components are required, to the contrary, various optional components are described to illustrate the wide variety of possible embodiments of the present disclosure.

[0024] In the following detailed description of embodiments of the present disclosure, reference is made to the accompanying drawings which form a part hereof, and which show, by way of illustration, specific embodiments in which the present disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, it being understood that other embodiments may be utilized and changes may be made without departing from the scope of the present disclosure. Accordingly, the following description is not to be construed in a limiting sense.

[0025] FIG. 1 illustrates an example architecture for lane tracking in an autonomous vehicle according to some embodiments of the present disclosure.

[0026] The architecture 100 includes an autonomous vehicle 101, a lane tracking system 103 and a lane boundary detection system 105, and sensors 1131 to 1133. n (also referred to as one or more sensors 113) and image capture device 115. By way of example, autonomous vehicle 101 may be a car, bus, truck, lorry, etc., integrated with electronic control units (ECUs) and systems capable of communicating over an on-board network of the autonomous vehicle. In some embodiments, lane boundary detection system 105 may be associated with lane tracking system 103 via a communications network (not shown in FIG. 1 ). The communications network may be at least one of a wired communications network and a wireless communications network. In some embodiments, both lane boundary detection system 105 and lane tracking system 103 may be configured at vehicle 101 to perform lane tracking of autonomous vehicle 101. In some other embodiments, both lane boundary detection system 105 and lane tracking system 103 may be externally associated with an ECU of autonomous vehicle 101 that performs lane tracking of autonomous vehicle 101. In yet other embodiments, one of the systems may be configured in autonomous vehicle 101 and the other of the system may be externally associated with an ECU of autonomous vehicle 101 that performs lane tracking for autonomous vehicle 101.

[0027] In some embodiments, the autonomous vehicle 101 may be configured with one or more sensors 113 and image capture devices 115. The autonomous vehicle 101 may sense its environment using one or more sensors 113, such as a laser, a light detection and ranging (LIDAR), a global positioning system (GPS), computer vision, etc. Additionally, an image capture device 115 may be attached to the autonomous vehicle 101 to capture image frames of the area in front of the autonomous vehicle 101. In some embodiments, the image capture device 115 may include, but is not limited to, a red-green-blue (RGB) camera, a monochrome camera, a depth camera, a 360-degree camera, a night vision camera, etc. In some embodiments, the autonomous vehicle 101 may be attached with two or more image capture devices 115. The image capture devices 115 may be attached to the area of ​​the autonomous vehicle 101 such that the area in front of the autonomous vehicle 101 is adequately covered in the image frames. For example, image capture device 115 may be mounted on autonomous vehicle 101, in the headlamp area of ​​autonomous vehicle 101, on an exterior rearview mirror, etc.

[0028] In some embodiments, the lane tracking system 103 is an artificial intelligence (AI)-based system and may be trained to track lanes of the autonomous vehicle 101 before deploying the lane tracking system 103 in a dynamic environment in which the autonomous vehicle 101 is traveling. In some embodiments, during the training phase, the lane tracking system 103 may receive ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to a plurality of lane boundary detection points from the lane boundary detection system 105. In some embodiments, the ground truth values ​​may refer to original values, in other words, information that is known to be real and is provided based on direct observation. However, the measurements are values ​​determined or predicted by the system and are not based on direct observation like the ground truth values. In some embodiments, the lane boundaries may be lines indicating the boundaries of the lane. Each lane may have a left lane boundary and a right lane boundary surrounding it. In some embodiments, the lane boundary detection points may be points indicating boundary areas of the road along which the autonomous vehicle 101 is traveling. In other words, the lane boundary detection points correspond to left and right lane boundaries of multiple lanes belonging to the road along which the autonomous vehicle 101 travels. A subset of consecutive lane boundary detection points and corresponding ground truth values ​​may be referred to as a ground truth set, and a subset of consecutive lane boundary detection points and corresponding measurements may be referred to as a measured set. The lane tracking system 103 may then use the ground truth set to generate ground truth clothoid points and the measured set to generate measured clothoid points. In some embodiments, during this training phase, the lane tracking system 103 may be trained to select the ground truth set and the measured set necessary to generate the ground truth clothoid points and the measured clothoid points, respectively. Clothoid points are generally spiral curves whose curvature varies linearly over the length of the arc, allowing for smooth steering wheel movement when the autonomous vehicle 101 is traveling on road segments with different horizontal curvatures.The lane tracking system 103 may then be trained to determine coefficient values ​​of clothoid parameters for the ground truth clothoid points and the measured clothoid points to model the lane boundaries of the lane in which the autonomous vehicle 101 is traveling. In some embodiments, the clothoid parameters may include, but are not limited to, the initial curvature (c) of the lane boundaries. o ), the rate of change of curvature of the lane boundary (c1), and the heading angle (β) relative to the direction of travel of the autonomous vehicle. In some embodiments, the initial curvature of the lane boundary (c o) may be defined as the first curvature angle of the lane determined in the image frame, the lane boundary curvature change rate (c1) may be defined as the rate at which the lane curvature changes in the image frame compared to the initial curvature, and the heading angle (β) may be defined as the lane curvature angle relative to the autonomous vehicle on that lane. To track the lane boundary, the lane tracking system 103 is then trained to determine Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points. In some embodiments, the lane tracking system 103 may determine the Kalman filter parameters using at least one neural network, such as an RNN or a long-short-term memory (LSTM) network. Having determined the Kalman filter parameters, the lane tracking system 103 may be trained to update the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters. The lane tracking system 103 may then be trained to reconstruct the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. In some embodiments, each reconstructed measured clothoid point enables lane tracking system 103 to track the lane boundaries of autonomous vehicle 101. Lane tracking system 103 may then determine a training error by calculating the difference between the reconstructed clothoid points and the corresponding ground truth set. During the training phase, lane tracking system 103 may minimize the determined training error in each cycle until the training error is below a predefined threshold.

[0029] The lane tracking system 103 trained in this manner can be used in a dynamic environment in which the autonomous vehicle 101 is traveling on a road. In some embodiments, the lane tracking system 103 may include a processor 107, an input / output (I / O) interface 109, and a memory 111, as shown in FIG. 1 . The I / O interface 109 of the lane tracking system 103 may receive measurements corresponding to a plurality of lane boundary detection points from the lane boundary detection system 105. The plurality of lane boundary detection points correspond to a left lane boundary and a right lane boundary of a lane in which the autonomous vehicle 101 is traveling. The processor 107 may generate a measured clothoid point based on a measured set including a subset of consecutive lane boundary detection points and the corresponding measurements. In some embodiments, the processor 107 may dynamically select a subset of consecutive lane boundary detection points to form the measured set. In some embodiments, the processor 107 may determine coefficient values ​​of clothoid parameters of the measured clothoid points to model the lane boundaries of the lane. The processor 107 may then determine Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane, and use the Kalman filter parameters to update the coefficient values ​​of the clothoid parameters determined for the measured clothoid points. Finally, the processor 107 may reconstruct the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. In some embodiments, each reconstructed measured clothoid point enables the lane tracking system 103 to track the lane boundaries of the autonomous vehicle 101.

[0030] FIG. 2A shows a detailed block diagram of a lane tracking system 103 for an autonomous vehicle, according to some embodiments of the disclosure.

[0031] In some implementations, lane tracking system 103 may include data 203 and module 205. As an example, data 203 is stored in memory 111 of lane tracking system 103, as shown in FIG. 2A. In one embodiment, data 203 may include training data 207, clothoid point data 209, Kalman filter parametric data 211, reconstructed data 213, and other data 215. In FIG. 2A, module 205 is described in detail herein.

[0032] In some embodiments, data 203 may be stored in memory 111 in the form of various data structures. Additionally, data 203 may be organized using a data model, such as a relational or hierarchical data model. Other data 215 may store data, including temporary data and files, generated by modules 205 to perform various functions of lane tracking system 103.

[0033] In some embodiments, training data 207 may include data used to train lane tracking system 103 for lane tracking of autonomous vehicle 101. For example, training data 207 may include, but is not limited to, ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to a plurality of lane boundary detection points, ground truth clothoid points and measured clothoid points generated using the ground truth set and the measured set, respectively, coefficient values ​​of Kalman filter parameters and clothoid parameters determined for the measured clothoid points, updated coefficient values ​​of the clothoid parameters, reconstructed measured clothoid points, and a training error used to train lane tracking system 103.

[0034] In some embodiments, clothoid point data 209 may include data related to clothoid points generated in a dynamic environment in which autonomous vehicle 101 is moving on a roadway. By way of example, and without limitation, clothoid point data 209 may include a measured clothoid point, a measurement set including a subset of multiple consecutive lane boundary detection points and the corresponding measurements used to generate the measured clothoid point, and coefficient values ​​of clothoid parameters determined for the measured clothoid point.

[0035] In some embodiments, the Kalman filter parametric data 211 may include, but is not limited to, Kalman filter parameters determined for coefficient values ​​of clothoid parameters determined for measured clothoid points using at least one neural network, which may include an RNN or a short-term memory (LSTM) network.

[0036] In some embodiments, at least one neural network may include a neural network with temporal memory. The neural network with temporal memory may be a specialized recurrent neural network (RNN) capable of learning long-term dependencies. At least one neural network may include an LSTM explicitly designed to avoid the problem of long-term dependencies. All RNNs have a chain of repeated neural network modules. In standard RNNs, this repeated module has a very simple structure, such as a single hyperbolic tangent layer. LSTMs also have this chain-like structure, but the repeated modules of LSTMs have a different structure than typical RNNs. Rather than having a single neural network layer, there are four neural networks that interact in a special way. LSTMs have the ability to remove or add information from the cell state, carefully controlled by structures called gates. Gates are a way to selectively pass information. For example, to protect and control the cell state, LSTMs have three such gates: (a) Input gates - determine which new information is stored in the cell state; (b) Forget gates - determine which information is discarded from the cell state; and (c) Output gates - determine which information is output.

[0037] In some embodiments, the reconstructed data 213 may include, but is not limited to, the reconstructed measured clothoid points and updated coefficient values ​​of the clothoid parameters determined for the measured clothoid points used in the reconstruction of the measured clothoid points.

[0038] In some embodiments, data 203 stored in memory 111 may be processed by modules 205 of lane tracking system 103. Modules 205 may be stored within memory 111. In one example, modules 205 communicatively coupled to processor 107 of lane tracking system 103 may also reside outside of memory 111 and be implemented in hardware, as shown in FIG. 2A . As used herein, the term module 205 may refer to an application specific integrated circuit (ASIC), electronic circuitry, processors (shared, dedicated, or group) and memory executing one or more software or firmware programs, combinatorial logic circuitry, and / or other suitable components that provide the described functionality.

[0039] In some embodiments, the modules 205 may include, for example, a receiving module 221, a coefficient value determination module 223, a Kalman filter determination module 225, a reconstruction module 227, a learning module 229, and other modules 231. The other modules 231 may be used to perform various miscellaneous functions of the lane tracking system 103. It should be understood that such aforementioned modules 205 may be represented as a single module or a combination of different modules.

[0040] The lane tracking system 103 may be trained to track the lane of the autonomous vehicle 101 before deploying the lane tracking system 103 in the dynamic environment in which the autonomous vehicle 101 is traveling.

[0041] In some embodiments, during the training phase, the receiving module 221 may receive ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system 105 associated with the lane tracking system 103. In some embodiments, the plurality of lane boundary detection points correspond to left and right lane boundaries of a plurality of lanes belonging to a road along which the autonomous vehicle 101 is traveling. In some embodiments, the lane boundary detection system 105 may determine the plurality of lane boundary detection points using lane data received from one or more sensors 113 configured on the autonomous vehicle 101 and / or image frames of the lanes received in real time. In some embodiments, the image frames of the lanes are received from an image capture device 115 associated with the autonomous vehicle 101. In some embodiments, the lane data and / or image frames of the lanes are obtained from a database configured to store lane data and image frames captured in real time. By way of example, the lane data may include, but is not limited to, lane markings, lane patterns, lane colors, number of lanes, etc. FIG. 2B shows an exemplary diagram illustrating an exemplary plurality of lane boundary detection points 233 arranged consecutively according to exemplary ground truth values. Because the ground truth values ​​corresponding to the exemplary plurality of lane boundary detection points 233 are original values, the white lines on the lanes in FIG. 2B are smooth lines that properly overlap the lanes of the road. FIG. 2C shows an exemplary diagram illustrating an exemplary plurality of lane boundary detection points 235 arranged consecutively according to exemplary measurement values. Because the measurement values ​​corresponding to the exemplary plurality of lane boundary detection points 235 are measured by a trained lane tracking system 103, the white lines on the lanes in FIG. 2B are not smooth and do not overlap the lanes of the road.

[0042] Further, in some embodiments, during the training phase, the coefficient value determination module 223 may select a subset of consecutive lane boundary detection points and corresponding ground truth values ​​as a ground truth set, and may select a subset of consecutive lane boundary detection points and corresponding measurements as a measured set. The coefficient value determination module 223 may use the ground truth set to generate ground truth clothoid points and use the measured set to generate measured clothoid points. In some embodiments, clothoid points are generally generated using a predefined number of consecutive lane boundary detection points. Thus, as the autonomous vehicle 101 moves along a lane of a road, the ground truth set and the measured set are successively selected, one after the other, to generate respective ground truth clothoid points and measured clothoid points. An exemplary ground truth set containing “N” ground truth lane boundary detection points may be shown as follows: [(x0,y0),(x1,y1),(x2,y2),……(x n-1 ,y n-1 )]

[0043] Similarly, an exemplary ground truth set containing "M" lane boundary detection points of measurements can be shown as follows: [(x0,y0),(x1,y1),(x2,y2),……(x m-1 ,y m-1 )]

[0044] Then, during the training phase, the coefficient value determination module 223 may be trained to determine coefficient values ​​of clothoid parameters for the ground truth clothoid points and the measured clothoid points to model the lane boundaries of the lane in which the autonomous vehicle 101 travels. In some embodiments, the clothoid parameters may include, but are not limited to, the initial curvature of the lane boundaries (c o ), the rate of change of curvature of the lane boundary (c1), and the heading angle (β) relative to the direction of travel of the autonomous vehicle. In some embodiments, the initial curvature of the lane boundary (c o) may be defined as the first curvature angle of the lane determined in the image frame, the lane boundary curvature change rate (c1) may be defined as the rate at which the lane curvature changes in the image frame compared to the initial curvature, and the heading angle (β) may be defined as the angle at which the autonomous vehicle is expected to move forward relative to the lane curvature.

[0045] In some embodiments, because clothoid points cannot be evaluated in closed form, the coefficient value determination module 223 may be trained to determine the ground truth clothoid points and the measured clothoid points using Equation 1 below:

number

[0046] In the above formula 1, - x and y refer to lane boundary detection points in either the ground truth set or the measured set; -c o refers to the initial curvature of the lane boundary, - c1 refers to the rate of change of curvature of the lane boundary, - β refers to the heading angle of the autonomous vehicle relative to the driving direction, -x offset refers to the initial lateral offset between the lane boundary and the autonomous vehicle 101 (ego vehicle).

[0047] In some embodiments, the coefficient value determination module 223 may be trained to determine coefficient values ​​for clothoid parameters of ground truth clothoid points using Equation 2 below:

number

[0048] In the above formula 2, A is as follows:

number

number

[0049] Using Equation 2 above, the coefficient value determination module 223 may also determine coefficient values ​​for the clothoid parameters of the measured clothoid points. In some embodiments, the coefficient values ​​for the clothoid parameters of the ground truth clothoid points and the measured clothoid points represent the state of the boundaries of the lane in which the autonomous vehicle 101 is traveling during the training phase.

[0050] The Kalman filter determination module 225 may then be trained to determine Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane. In some embodiments, the Kalman filter determination module 225 may determine the Kalman filter parameters using at least one neural network, such as an RNN or a long-short-term memory (LSTM) network. The Kalman filter determination module 225 may determine the Kalman filter parameters using one or more neural networks. In some embodiments, the Kalman filter determination module 225 may initially provide the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as input to a first neural network. In some embodiments, the first neural network may be trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using past measured sets and ground truth sets. The Kalman filter determination module 225 may then use the first neural network to determine a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points. The first neural network may be capable of learning long-term dependencies. The first neural network may include an RNN or an LSTM network. Upon determining "R", the Kalman filter determination module 225 determines the state transition (Y ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another based on the speed of the autonomous vehicle 101 moving along the lane and the time difference between consecutive image frames. p ) In some embodiments, state transitions may be predicted using the matrix:

number

[0051] In the above matrix, "v" refers to the speed of the autonomous vehicle, "Δt" refers to the time difference between successive image frames.

[0052] The Kalman filter determination module 225 may then determine a process noise covariance matrix (Q) using the predicted state transitions as input to a second neural network. In some embodiments, the second neural network may also be trained using past ego vehicle speed values ​​and time difference values. The second neural network may be capable of learning long-term dependencies. The second neural network may include an RNN or LSTM network. Using the determined process noise covariance matrix (Q), the Kalman filter determination module 225 may determine the error covariance (P p ) Finally, the Kalman filter decision module 225 may predict the predicted state transition (Y p ) and covariance (P p ), the determined measurement noise covariance matrix (R), and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points, the Kalman filter may determine Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points. In some embodiments, broadly speaking, the Kalman filter may include, but is not limited to, a state vector (y), a state transition matrix (f), a state error covariance matrix (P), a process noise covariance matrix (Q), a Kalman gain (K), a measurement noise covariance matrix (R), and measurements (z) (also referred to as measurements corresponding to a plurality of lane boundary detection points) at time (t). As described above, the Kalman filter determination module 225 may learn Q and R using a first neural network and a second neural network, respectively. Equations 1-5 below illustrate the determination of the remaining portion of the Kalman filter parameters. Equations 1 and 2 relate to predicting the Kalman filter parameters, and Equations 3-5 relate to updating the Kalman filter parameters.

[0053] In the following formula 1,

number

number

number

[0054] In the following formula 2,

number

number

number

[0055] In the following formula 3,

number

number

[0056] In the following formula 4,

number

number

number

[0057] In the following formula 5,

number

number

number

[0058] Once the Kalman filter parameters are determined, the reconstruction module 227 can be trained to use the corresponding Kalman filter parameters to update the coefficient values ​​of the clothoid parameters determined for the measured clothoid points. As an example, the coefficient values ​​of the clothoid parameters updated based on the Kalman filter parameters can be shown as follows:

number

[0059] In the above example, KF refers to the Kalman filter parameters, and the clothoid parameters prefixed with "KF" indicate that the determined Kalman filter parameters are applied to the clothoid parameters to update the coefficient values ​​of the clothoid parameters.

[0060] In some embodiments, upon updating the coefficient values ​​of the clothoid parameters determined for the measured clothoid points, the reconstruction module 227 may be trained to reconstruct the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. In some embodiments, each reconstructed measured clothoid point enables the lane tracking system 103 to track the lane boundaries of the autonomous vehicle 101. In some embodiments, the reconstruction module 227 may add an initial lateral offset between the lane boundaries and the ego vehicle to the reconstructed measured clothoid points. As an example, an exemplary tracked and modeled lane 237 formed based on the reconstructed clothoid points is shown in FIG. 2D . In a further example, FIG. 2E illustrates multiple exemplary tracked lanes belonging to a road on which the autonomous vehicle 101 is traveling. In FIG. 2E , the line indicated by reference numeral 235 represents initial measurements corresponding to the exemplary multiple lane boundary detection points 235, and the line indicated by reference numeral 237 represents the exemplary tracked and modeled lane 237 formed based on the reconstructed measured clothoid points. Thus, in this disclosure, the lane tracking system 103 not only tracks the ego-lane, meaning the left and right boundary lanes of the lane in which the autonomous vehicle 101 is traveling, but also tracks other lanes of the road, as shown in FIG. 2E.

[0061] Additionally, the learning module 229 may determine a training error by calculating the difference between the reconstructed clothoid points and the corresponding ground truth set. By calculating the difference between the reconstructed clothoid points and the corresponding ground truth set, the learning module 229 can enhance error minimization, which leads to accurately determining the reconstructed measured clothoid points for lane tracking of the autonomous vehicle 101 when deployed in a dynamic environment. During the training phase, the learning module 229 may minimize the determined training error in each cycle until the training error is below a predefined threshold.

[0062] In some embodiments, the lane tracking system 103 trained in this manner may be used in a dynamic environment where the autonomous vehicle 101 is moving along a road.

[0063] In some embodiments, in a dynamic environment, the receiver module 221 may receive measurements corresponding to multiple lane boundary detection points from the lane boundary detection system 105. The multiple lane boundary detection points correspond to a left lane boundary and a right lane boundary of the lane in which the autonomous vehicle 101 is currently traveling.

[0064] The coefficient value determination module 223 may then generate measured clothoid points based on a measured set including a subset of consecutive lane boundary detection points and the corresponding measurements. In some embodiments, the coefficient value determination module 223 may dynamically select a subset of consecutive lane boundary detection points to form the measured set as the autonomous vehicle 101 travels along a road. The coefficient value determination module 223 may then determine coefficient values ​​for clothoid parameters of the measured clothoid points to model the lane boundaries of the lane.

[0065] Additionally, in some embodiments, the Kalman filter determination module 225 may determine Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane. In some embodiments, the Kalman filter determination module 225 may determine Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane. p ) and covariance (P p ), based on the determined measurement noise covariance matrix (R) as well as the coefficient values ​​of the clothoid parameters determined for the measured clothoid points, Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points can be determined.

[0066] The reconstruction module 227 may then use the Kalman filter parameters to update the coefficient values ​​of the clothoid parameters determined for the measured clothoid points. Finally, the reconstruction module 227 may reconstruct the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters. In some embodiments, each reconstructed measured clothoid point enables the lane tracking system 103 to track the lane boundaries of the autonomous vehicle 101.

[0067] FIG. 3A shows a flowchart illustrating a method for training a lane tracking system of an autonomous vehicle according to some embodiments of the present disclosure.

[0068] 3A, method 300a includes one or more blocks illustrating a method for training a lane tracking system of an autonomous vehicle 101. Method 300a may be described generally in the context of computer-executable instructions. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, procedures, modules, and functions that perform functions or implement abstract data types.

[0069] The order in which method 300a is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement method 300a. Additionally, individual blocks can be deleted from the method without departing from the spirit and scope of the subject matter described herein. Furthermore, method 300a can be implemented by any suitable hardware, software, firmware, or combination thereof.

[0070] At block 301, method 300a may include receiving, by a processor 107 of the lane tracking system 103 during a training phase, ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system 105 associated with the lane tracking system 103. In some embodiments, the plurality of lane boundary detection points may be determined using lane data received from one or more sensors 113 configured on the vehicle 101 and / or image frames of the lane received in real time. In some embodiments, the image frames of the lane are received from an image capture device 115 associated with the autonomous vehicle 101. In some other embodiments, the lane data and / or image frames of the lane are obtained from a database configured to store lane data and image frames captured in real time. The database may be associated with the lane tracking system 103.

[0071] At block 303, method 300a may include, during the training phase, determining, by processor 107, coefficient values ​​of a clothoid parameter for ground truth clothoid points and measured clothoid points formed using the ground truth set and the measured set, respectively, to model the lane boundaries of the lane. In some embodiments, processor 107 may select a subset of consecutive lane boundary detection points and corresponding ground truth values ​​to form the ground truth set, and may select a subset of consecutive lane boundary detection points and corresponding measurements to form the measured set. In some embodiments, the number of consecutive lane boundary detection points selected to form the ground truth set and the measured set may be predefined. However, the selection is made by processor 107 in real time. In some other embodiments, the number of consecutive lane boundary detection points selected to form the ground truth set and the measured set may be determined per requirement by processor 107 for each image frame. In some embodiments, the clothoid parameter may be determined based on, but is not limited to, an initial curvature (co ), the rate of change of curvature of the lane boundary (c1), and the heading angle (β) relative to the vehicle's direction of travel.

[0072] At block 305, method 300a includes, during a training phase, determining, by processor 107, Kalman filter parameters for coefficient values ​​of clothoid parameters determined for measured clothoid points to track lane boundaries of the lane. In some embodiments, processor 107 may determine the Kalman filter parameters using a neural network, such as an RNN or a long short-term memory (LSTM) network.

[0073] At block 307, the method 300a may include, during the training phase, updating, by the processor 107, coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters.

[0074] At block 309, the method 300a includes, during the training phase, reconstructing, by the processor 107, the measured clothoid points using corresponding updated coefficient values ​​of the clothoid parameters. In some embodiments, each reconstructed measured clothoid point enables the lane tracking system 103 to track the lane boundaries of the autonomous vehicle 101.

[0075] At block 311, method 300a includes, during the training phase, processor 107 minimizing, in each cycle, a training error based on the difference between the reconstructed measured clothoid points and the corresponding ground truth set until the training error is below a predefined threshold. In this way, for each cycle, processor 107 reduces the error in tracking lanes using lane tracking system 103 during training. In some embodiments, processor 107 may add an initial lateral offset between the lane boundary and autonomous vehicle 101 to the reconstructed measured clothoid points.

[0076] FIG. 3B shows a flowchart illustrating a method for lane tracking for an autonomous vehicle according to some embodiments of the present disclosure.

[0077] 3A, method 300b includes one or more blocks illustrating a method for lane tracking for an autonomous vehicle 101. Method 300b may be described generally in the context of computer-executable instructions. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, procedures, modules, and functions that perform functions or implement abstract data types.

[0078] The order in which method 300b is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement method 300b. Additionally, individual blocks can be deleted from the method without departing from the spirit and scope of the subject matter described herein. Furthermore, method 300b can be implemented by any suitable hardware, software, firmware, or combination thereof.

[0079] At block 313, method 300b may include receiving, by a processor 107 of lane tracking system 103, measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system 105 associated with lane tracking system 103 in a dynamic environment in which autonomous vehicle 101 is traveling along a road. In some embodiments, the plurality of lane boundary detection points are determined using lane data received from one or more sensors 113 configured on autonomous vehicle 101 and / or image frames of the lane received in real time. In some embodiments, processor 107 may receive image frames of the lane from an image capture device 115 associated with autonomous vehicle 101. In some embodiments, processor 107 may obtain the lane data and / or image frames of the lane from a database configured to store lane data and image frames captured in real time. The database may be associated with lane tracking system 103.

[0080] At block 315, method 300b may include determining, by processor 107, coefficient values ​​of clothoid parameters for measured clothoid points formed using each measurement set to model the lane boundary of the lane in a dynamic environment. In some embodiments, processor 107 may select a subset of consecutive lane boundary detection points and corresponding measurements to form the measured set. In some embodiments, the number of consecutive lane boundary detection points selected to form the measured set may be predefined. However, the selection is made by processor 107 in real time. In some other embodiments, the number of consecutive lane boundary detection points selected to form the measured set may be determined per requirement by processor 107 for each image frame. In some embodiments, the clothoid parameters may be determined based on, but are not limited to, the initial curvature of the lane boundary (c o ), the rate of change of curvature of the lane boundary (c1), and the heading angle (β) relative to the vehicle's direction of travel.

[0081] At block 317, method 300b may include determining, by processor 107, Kalman filter parameters for coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track lane boundaries of the lane in a dynamic environment. In some embodiments, processor 107 may determine the Kalman filter parameters using a neural network capable of learning long-term dependencies, such as an RNN or a long short-term memory (LSTM) network.

[0082] At block 319, the method 300b may include updating, by the processor 107, coefficient values ​​of the determined clothoid parameters for the measured clothoid points using the corresponding Kalman filter parameters in a dynamic environment.

[0083] At block 321, method 300b may include reconstructing, by processor 107, measured clothoid points in a dynamic environment using corresponding updated coefficient values ​​of the clothoid parameters. In some embodiments, each reconstructed measured clothoid point enables lane tracking system 103 to track lane boundaries of autonomous vehicle 101.

[0084] FIG. 4 is a block diagram of an exemplary computer system for implementing embodiments consistent with this disclosure.

[0085] In some embodiments, FIG. 4 illustrates a block diagram of an exemplary computer system 400 for implementing embodiments consistent with the present invention. In some embodiments, computer system 400 may be lane tracking system 103 for lane tracking in an autonomous vehicle, as shown in FIG. 4. Computer system 400 may include a central processing unit (“CPU” or “processor”) 402. Processor 402 may include at least one data processor for executing program components for executing user- or system-generated business processes. A user may include a person, a person using a device such as those included in the present invention, or such a device itself. Processor 402 may include dedicated processing units such as an integrated system (bus) controller, a memory management control unit, a floating-point unit, a graphics processing unit, a digital signal processing unit, etc.

[0086] The processor 402 may be placed in communication with input devices 411 and output devices 412 via an I / O interface 401. The I / O interface 401 may employ communication protocols / methods including, but not limited to, audio, analog, digital, stereo, IEEE-1394, serial bus, universal serial bus (USB), infrared, PS / 2, BNC, coaxial, component, composite, digital visual interface (DVI), high-definition multimedia interface (HDMI), radio frequency (RF) antenna, S-Video, video graphics array (VGA), IEEE 802.n / b / g / n / x, Bluetooth, cellular (e.g., code division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long-term evolution (LTE), WiMax, etc.).

[0087] Using the I / O interface 401 , the computer system 400 can communicate with input devices 411 and output devices 412 .

[0088] In some embodiments, the processor 402 may be disposed in communication with a communications network 409 via a network interface 403. The network interface 403 is capable of communicating with the communications network 409. The network interface 403 may employ a connection protocol including, but not limited to, Direct Connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), Transmission Control Protocol / Internet Protocol (TCP / IP), Token Ring, IEEE 802.11a / b / g / n / x, etc. Using the network interface 403 and the communications network 409, the computer system 400 may communicate with the lane boundary detection system 105, one or more sensors 113, and the image capture device 115. In some embodiments, the lane tracking system 103 may also be associated with a database (not shown in FIG. 4 ). The communications network 409 may be implemented as one of different types of networks, such as an intranet or local area network (LAN), a closed area network (CAN), etc., and within an autonomous vehicle. The communications network 409 may be either a dedicated network or a shared network, which represents a collection of different types of networks that use various protocols, such as HyperText Transfer Protocol (HTTP), CAN protocol, Transmission Control Protocol / Internet Protocol (TCP / IP), Wireless Application Protocol (WAP), etc., to communicate with each other. Additionally, the communications network 409 may include various network devices, including routers, bridges, servers, computing devices, storage devices, etc. The one or more sensors 113 may include, but are not limited to, Light Detection and Ranging (LIDAR) systems, Global Positioning Systems (GPS), lasers, etc. In some embodiments, the processor 402 may be disposed in communication with memory 405 (e.g., RAM, ROM, etc., not shown in FIG. 4 ) via a storage interface 404.The storage interface 404 may employ connection protocols such as Serial Advanced Technology Attachment (SATA), Integrated Drive Electronics (IDE), IEEE-1394, Universal Serial Bus (USB), Fibre Channel, Small Computer System Interface (SCSI), etc. to connect to memory 405, including, but not limited to, memory drives, removable disk drives, etc. The memory drives may further include drum, magnetic disk drives, magneto-optical drives, optical drives, Redundant Array of Independent Disks (RAID), solid-state memory devices, solid-state drives, etc.

[0089] Memory 405 may store a collection of program or database components, including but not limited to a user interface 406, an operating system 407, a web browser 408, etc. In some embodiments, computer system 400 may store user / application data, such as data, variables, records, etc., as described in the present invention. Such databases may be implemented as fault-tolerant, relational, scalable, and secure databases, such as Oracle or Sybase.

[0090] Operating system 407 may facilitate resource management and operation of computer system 400. Examples of operating systems 407 include, but are not limited to, APPLE® MACINTOSH® OS X®, UNIX®, UNIX-like system distributions (e.g., BERKELEY SOFTWARE DISTRIBUTION® (BSD), FREEBSD®, NETBSD®, OPENBSD, etc.), LINUX® distributions (e.g., RED HAT®, UBUNTU®, KUBUNTU®, etc.), IBM® OS / 2®, MICROSOFT® WINDOWS® (XP®, VISTA® / 7 / 8, 10, etc.), APPLE® IOS®, GOOGLE® ANDROID®, BLACKBERRY® OS, etc. The user interface 406 may facilitate the display, execution, interaction, manipulation, or operation of program components through textual or graphical features. For example, the user interface 406 may provide computer interaction interface elements, such as cursors, icons, checkboxes, menus, scrollers, windows, widgets, and the like, on a display system operatively connected to the computer system 400. Graphical user interfaces (GUIs) may be employed, including, but not limited to, Apple® Macintosh® operating system Aqua®, IBM® OS / 2®, Microsoft® Windows® (e.g., Aero, Metro, etc.), web interface libraries (e.g., ActiveX®, Java®, Javascript®, AJAX, HTML, Adobe® Flash®, etc.), and the like.

[0091] In some embodiments, computer system 400 may implement a web browser 408 stored program component. Web browser 408 may be a hypertext browsing application such as MICROSOFT® INTERNET EXPLORER®, GOOGLE™ CHROME™, MOZILLA® FIREFOX®, APPLE® SAFARI®, etc. Secure web browsing may be provided using Secure Hypertext Transfer Protocol (HTTPS), Secure Sockets Layer (SSL), Transport Layer Security (TLS), etc. Web browser 408 may utilize features such as AJAX, DHTML, ADOBE® FLASH®, JAVASCRIPT®, JAVA®, application programming interfaces (APIs), etc. In some embodiments, computer system 400 may implement a mail server stored program component. The mail server may be an Internet mail server such as Microsoft Exchange. The mail server may utilize functionality such as Active Server Pages (ASP), ACTIVEX®, ANSI® C++ / C#, MICROSOFT®, .NET, CGI script, JAVA®, JAVASCRIPT®, PERL®, PHP, PYTHON®, WEBOBJECTS®, etc. The mail server may utilize communication protocols such as Internet Message Access Protocol (IMAP), Messaging Application Programming Interface (MAPI), MICROSOFT® exchange, Post Office Protocol (POP), Simple Mail Transfer Protocol (SMTP), etc. In some embodiments, computer system 400 may implement a mail client storage program component.The email client can be an email viewing application such as APPLE® MAIL, MICROSOFT® ENTOURAGE®, MICROSOFT® OUTLOOK®, MOZILLA® THUNDERBIRD®, or the like.

[0092] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present invention. A computer-readable storage medium refers to any type of physical memory capable of storing processor-readable information or data. Thus, a computer-readable storage medium may store instructions executed by one or more processors, including instructions that cause a processor(s) to perform steps or stages consistent with embodiments described herein. The term "computer-readable medium" should be understood to include tangible items and exclude carrier waves and transient signals, i.e., non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard drives, compact disc (CD) ROMs, digital video discs (DVDs), flash drives, disks, and any other known physical storage medium.

[0093] A description of an embodiment having several components in communication with each other does not imply that all such components are required. To the contrary, various optional components are described to illustrate the wide variety of possible embodiments of the present invention. Where a single device or article is described herein, it will be apparent that two or more devices / articles (whether or not they cooperate) may be used in place of the single device / article. Similarly, where two or more devices or articles are described herein (whether or not they cooperate), it will be apparent that a single device / article may be used in place of two or more devices or articles, or that a different number of devices / articles may be used in place of the number of devices or programs shown. The functionality and / or features of a device may alternatively be embodied by one or more other devices not explicitly described as having such functionality / features. Thus, other embodiments of the present invention need not include a device itself.

[0094] This specification has described a method and system for lane tracking for an autonomous vehicle 101. The illustrated steps are presented to explain the illustrated exemplary embodiment, and it is expected that ongoing technological developments will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration and not limitation. Furthermore, the boundaries of functional building blocks have been arbitrarily defined herein for convenience of description. Alternative boundaries may be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to those skilled in the relevant art(s) based on the teachings contained herein. Such alternatives are within the scope and spirit of the disclosed embodiments. Furthermore, the words “comprise,” “have,” “contain,” and “include,” and other similar forms are intended to be equivalent in meaning and open-ended in that the items following any one of these words are not intended to be an exhaustive listing of such items or to be limited to only the listed item or items. It should also be noted that as used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise.

[0095] Finally, the language used herein has been chosen primarily for ease of reading and teaching purposes, and not to delineate or limit the subject matter of the present invention. Accordingly, it is intended that the scope of the invention be limited not by this detailed description, but by the claims that issue on an application based thereon. Accordingly, the embodiments of the present invention are intended to illustrate, but not limit, the scope of the invention, which is set forth in the following claims. The present application relates to the invention described in the claims, but also includes the following as other aspects. 1. A method for training a lane tracking system (103) of an autonomous vehicle (101), comprising: receiving, by a lane tracking system (103), from a lane boundary detection system (105) associated with said lane tracking system (103), ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to said plurality of lane boundary detection points; determining, by the lane tracking system (103), coefficient values ​​of clothoid parameters for ground truth clothoid points and measured clothoid points formed using a ground truth set and a measured set, respectively, to model lane boundaries of a lane, wherein the ground truth set includes a subset of consecutive lane boundary detection points and the corresponding ground truth values, and the measured set includes a subset of consecutive lane boundary detection points and the corresponding measured values; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane by the lane tracking system (103), wherein the Kalman filter parameters are determined using at least one neural network; updating, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing, by the lane tracking system (103), the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); and minimizing, in each cycle, a training error based on the difference between the reconstructed measured clothoid points and the corresponding ground truth set, until the training error is below a predefined threshold, by the lane tracking system (103). A method comprising: 2. 2. The method according to claim 1, wherein the plurality of lane boundary detection points correspond to a left lane boundary and a right lane boundary of the lane. 3. 3. The method of claim 1 or 2, further comprising dynamically selecting, by the lane tracking system (103), the ground truth set and the measured set required to generate the ground truth clothoid points and the measured clothoid points, respectively. 4. 4. The method of any one of claims 1 to 3, further comprising determining the plurality of lane boundary detection points using at least one of lane data received from one or more sensors (113) configured in the vehicle and image frames of the lane received in real time, wherein the image frames of the lane are received from an image capture device (115) associated with the autonomous vehicle (101). 5. 5. The method of claim 4, comprising obtaining at least one of the lane data and the image frames of the lane from a database configured to store the lane data and the image frames captured in real time. 6. The clothoid parameters are the initial curvature of the lane boundary (c o ), the rate of change of curvature of the lane boundary (c 1 6. The method according to any one of 1 to 5 above, comprising: a heading angle (β) relative to the vehicle's traveling direction; 7. 7. The method according to any one of 1 to 6 above, wherein the at least one neural network comprises a neural network with temporal memory. 8. 8. The method according to any one of 1 to 7 above, wherein the at least one neural network comprises a recurrent neural network. 9. 9. The method according to any one of 1 to 8 above, wherein the at least one neural network comprises a long-short-term memory neural network. 10. Determining the Kalman filter parameters using the at least one neural network includes: providing, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as input to a first neural network, the first neural network being trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using a past measured set and a ground truth set; determining, by the lane tracking system (103), a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the first neural network; The lane tracking system (103) determines a state transition (Y ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another based on the speed of the autonomous vehicle (101) moving along the lane and the time difference between successive image frames. p ) and determining, by the lane tracking system (103), a process noise covariance matrix (Q) using the predicted state transitions as inputs to a second neural network, the second neural network being trained using past autonomous vehicle (101) speed and time difference values; The lane tracking system (103) uses the determined process noise covariance matrix (Q) to calculate the error covariance (P p ) and The lane tracking system (103) determines the predicted state transition (Y p ) and covariance (P p ), determining the Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points based on the determined measurement noise covariance matrix (R) and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; 10. The method according to any one of 1 to 9 above, comprising: 11. 11. The method of any one of claims 1 to 10, further comprising adding, by the lane tracking system (103), an initial lateral offset between the lane boundary and the autonomous vehicle (101) to the reconstructed measured clothoid point. 12. A method for lane tracking for an autonomous vehicle (101), comprising: receiving, by a lane tracking system (103), measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system (105) associated with said lane tracking system (103); determining, by the lane tracking system (103), coefficient values ​​of clothoid parameters for measured clothoid points formed using a measurement set to model lane boundaries of a lane, the measurement set including a subset of consecutive lane boundary detection points and the corresponding measurements; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane by the lane tracking system (103), wherein the Kalman filter parameters are determined using at least one neural network; updating, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing, by the lane tracking system (103), the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); A method comprising: 13. 13. The method of claim 12, wherein the at least one neural network comprises a neural network with temporal memory. 14. 14. The method of claim 12 or 13, wherein the at least one neural network comprises a recurrent neural network. 15. 15. The method according to any one of claims 12 to 14, wherein said at least one neural network comprises a long-short-term memory neural network. 16. A lane tracking system (103) for an autonomous vehicle (101), comprising: a processor (107); a memory (111) communicatively coupled to the processor (107), the memory (111) storing processor (107) instructions that, when executed, cause the processor (107) to train the lane tracking system (103); and for training, the processor (107) receiving, from a lane boundary detection system (105) associated with the lane tracking system (103), ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to the plurality of lane boundary detection points; determining coefficient values ​​of clothoid parameters for ground truth clothoid points and measured clothoid points formed using a ground truth set and a measured set, respectively, to model lane boundaries of a lane, wherein the ground truth set includes a subset of consecutive lane boundary detection points and the corresponding ground truth values, and the measured set includes a subset of consecutive lane boundary detection points and the corresponding measurement values; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane, the Kalman filter parameters being determined using at least one neural network; updating the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); In each cycle, minimizing a training error based on the difference between the reconstructed measured clothoid points and the corresponding ground truth set until the training error is below a predefined threshold. a lane tracking system (103) configured to: 17. 17. The lane tracking system (103) of claim 16, wherein the plurality of lane boundary detection points correspond to a left lane boundary and a right lane boundary of the lane. 18. 18. The lane tracking system (103) of claim 16 or 17, wherein the processor (107) dynamically selects the ground truth set and the measured set required to generate the ground truth clothoid points and the measured clothoid points, respectively. 19. A lane tracking system (103) as described in any one of claims 16 to 18, wherein the plurality of lane boundary detection points are determined using at least one of lane data received from one or more sensors (113) configured in the vehicle and image frames of the lane received in real time, the image frames of the lane being received from an image capture device (115) associated with the autonomous vehicle (101). 20. A lane tracking system (103) as described in claim 19, wherein the processor (107) obtains at least one of the lane data and the image frames of the lane from a database configured to store the lane data and the image frames captured in real time. 21. The clothoid parameters are the initial curvature of the lane boundary (c o ), the rate of change of curvature of the lane boundary (c 1 21. The lane tracking system (103) according to any one of 16 to 20 above, including a heading angle (β) relative to the vehicle's traveling direction. 22. To determine the Kalman filter parameters using the at least one neural network, the processor (107) providing the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as inputs to a first neural network, the first neural network being trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using past measured sets and a ground truth set; using the first neural network to determine a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; A state transition (Y ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another image frame is calculated based on the speed of the autonomous vehicle (101) moving along the lane and the time difference between successive image frames. p ) and determining a process noise covariance matrix (Q) using the predicted state transitions as inputs to a second neural network, the second neural network being trained using past autonomous vehicle (101) velocity and time difference values; The determined process noise covariance matrix (Q) is used to calculate the error covariance (P p ) and The predicted state transition (Y p ) and covariance (P p ), determining the Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points based on the determined measurement noise covariance matrix (R) and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; 22. The lane tracking system (103) according to any one of 16 to 21 above, configured to perform the following. 23. 23. The lane tracking system (103) of any one of claims 16 to 22, wherein the processor (107) is further configured to add an initial lateral offset between the lane boundary and the autonomous vehicle (101) to the reconstructed measured clothoid point. 24. 24. A lane tracking system (103) according to any one of claims 16 to 23, wherein said at least one neural network comprises a neural network with temporal memory. 25、 25. The lane tracking system (103) according to any one of claims 16 to 24, wherein the at least one neural network comprises a recurrent neural network. 26. 26. The lane tracking system (103) according to any one of claims 16 to 25, wherein the at least one neural network includes a long-short-term memory neural network. 27. A lane tracking system (103) for an autonomous vehicle (101), comprising: a processor (107); a memory (111) communicatively coupled to the processor (107), the memory (111) being configured, upon execution, to cause the processor (107) to: receiving measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system (105) associated with the lane tracking system (103); determining coefficient values ​​of clothoid parameters for measured clothoid points formed using a measurement set to model lane boundaries of a lane, the measurement set including a subset of consecutive lane boundary detection points and the corresponding measurements; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane, the Kalman filter parameters being determined using at least one neural network; updating the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); a memory (111) for storing instructions for a processor (107) to perform the a lane tracking system (103) including: 28. 28. A lane tracking system (103) as described in claim 27, wherein said at least one neural network includes a neural network with temporal memory. 29. 29. A lane tracking system (103) according to claim 27 or 28, wherein the at least one neural network comprises a recurrent neural network. 30. 30. The lane tracking system (103) according to any one of claims 27 to 29, wherein the at least one neural network includes a long-short-term memory neural network. [Explanation of symbols]

[0096] 100 Architecture 101 Autonomous Vehicles 103 Lane Tracking System 105 Lane Boundary Detection System 107 processors 109 I / O Interface 111 memory 113 One or more sensors 115 Image capture device 203 Data 205 modules 207 Training Data 209 Clothoid Point Data 211 Kalman Filter Parametric Data 213 Reconstructed Data 215 other data 221 Receiver Module 223 Coefficient Value Determination Module 225 Kalman Filter Decision Module 227 Reconstruction Module 229 Learning Modules 231 other modules 233 Exemplary Multiple Lane Boundary Detection Points Corresponding to Measurements 235 Exemplary Multiple Lane Boundary Detection Points Corresponding to Ground Truth Values 237 Example lanes tracked and modeled based on reconstructed clothoid points 400 Exemplary Computer System 401 I / O Interfaces of an Exemplary Computer System 402 Processor of an Exemplary Computer System 403 Network Interface 404 Storage Interface 405 Memory of an Exemplary Computer System 406 User Interface 407 Operating System 408 Web Browser 409 Communication Network 411 Input Devices 412 output devices

Claims

1. A method for training a lane tracking system (103) of an autonomous vehicle (101), comprising: receiving, by a lane tracking system (103), from a lane boundary detection system (105) associated with said lane tracking system (103), ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to said plurality of lane boundary detection points; determining, by the lane tracking system (103), coefficient values ​​of clothoid parameters for ground truth clothoid points and measured clothoid points formed using a ground truth set and a measured set, respectively, to model lane boundaries of a lane, wherein the ground truth set includes a subset of consecutive lane boundary detection points and the corresponding ground truth values, and the measured set includes a subset of consecutive lane boundary detection points and the corresponding measured values; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane by the lane tracking system (103), wherein the Kalman filter parameters are determined using at least one neural network; updating, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing, by the lane tracking system (103), the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); and minimizing, in each cycle, a training error based on the difference between the reconstructed measured clothoid points and the corresponding ground truth set, until the training error is below a predefined threshold, by the lane tracking system (103). In a method comprising: Determining the Kalman filter parameters using the at least one neural network comprises the steps of: providing, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as input to a first neural network, the first neural network being trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using a set of past measurements and a ground truth set; determining, by the lane tracking system (103), using the first neural network, a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; predicting, by the lane tracking system (103), a state transition (Y p ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another based on the speed of the autonomous vehicle (101) moving along the lane and the time difference between successive image frames; determining, by the lane tracking system (103), a process noise covariance matrix (Q) using the predicted state transitions as inputs to a second neural network, the second neural network being trained using past autonomous vehicle (101) speed and time difference values; predicting, by the lane tracking system (103), an error covariance (P p ) of the predicted state transition using the determined process noise covariance matrix (Q); determining, by the lane tracking system (103), the Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points based on the predicted state transitions (Y p ) and covariances (P p ), the determined measurement noise covariance matrix (R), and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; Including, A method characterized by:

2. The method of claim 1 , wherein the plurality of lane boundary detection points correspond to a left lane boundary and a right lane boundary of the lane.

3. 3. The method of claim 1 or 2, comprising dynamically selecting, by the lane tracking system (103), the ground truth sets and the measured sets required to generate the ground truth clothoid points and the measured clothoid points, respectively.

4. 3. The method of claim 1 or 2, comprising determining the plurality of lane boundary detection points using at least one of lane data received from one or more sensors (113) configured in the autonomous vehicle (101) and image frames of the lane received in real time, the image frames of the lane being received from an image capture device (115) associated with the autonomous vehicle (101).

5. 5. The method of claim 4, comprising obtaining the at least one of the lane data and the image frames of the lane from a database configured to store the lane data and the image frames captured in real time.

6. The clothoid parameters are the initial curvature of the lane boundary (c ｏ ), the rate of change of curvature of the lane boundary (c １ 3. The method of claim 1, wherein the angle of the autonomous vehicle relative to the direction of travel is a heading angle (β).

7. The method of claim 1 or 2, wherein the at least one neural network comprises a neural network with temporal memory.

8. The method of claim 1 or 2, wherein the at least one neural network comprises a recurrent neural network.

9. The method of claim 1 or 2, wherein the at least one neural network comprises a long short-term memory neural network.

10. 3. The method of claim 1, further comprising adding, by the lane tracking system (103), an initial lateral offset between the lane boundary and the autonomous vehicle (101) to the reconstructed measured clothoid point.

11. A method for lane tracking for an autonomous vehicle (101), comprising: receiving, by a lane tracking system (103), measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system (105) associated with said lane tracking system (103); determining, by the lane tracking system (103), coefficient values ​​of clothoid parameters for measured clothoid points formed using a measurement set to model lane boundaries of a lane, the measurement set including a subset of consecutive lane boundary detection points and the corresponding measurements; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane by the lane tracking system (103), wherein the Kalman filter parameters are determined using at least one neural network; updating, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing, by the lane tracking system (103), the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); In a method comprising: Determining the Kalman filter parameters using the at least one neural network comprises the steps of: providing, by the lane tracking system (103), the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as input to a first neural network, the first neural network being trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using a set of past measurements and a ground truth set; determining, by the lane tracking system (103), using the first neural network, a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; predicting, by the lane tracking system (103), a state transition (Y p ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another based on the speed of the autonomous vehicle (101) moving along the lane and the time difference between successive image frames; determining, by the lane tracking system (103), a process noise covariance matrix (Q) using the predicted state transitions as inputs to a second neural network, the second neural network being trained using past autonomous vehicle (101) speed and time difference values; predicting, by the lane tracking system (103), an error covariance (P p ) of the predicted state transition using the determined process noise covariance matrix (Q); determining, by the lane tracking system (103), the Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points based on the predicted state transitions (Y p ) and covariances (P p ), the determined measurement noise covariance matrix (R), and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; Including, A method characterized by:

12. The method of claim 11 , wherein the at least one neural network comprises a neural network with temporal memory.

13. 13. The method of claim 11 or 12, wherein the at least one neural network comprises a recurrent neural network.

14. 13. The method of claim 11 or 12, wherein the at least one neural network comprises a long short-term memory neural network.

15. A lane tracking system (103) for an autonomous vehicle (101), comprising: A processor (107); a memory (111) communicatively coupled to the processor (107), the memory (111) storing processor (107) instructions that, when executed, cause the processor (107) to train the lane tracking system (103); and for training, the processor (107) receiving, from a lane boundary detection system (105) associated with the lane tracking system (103), ground truth values ​​corresponding to a plurality of lane boundary detection points and measurements corresponding to the plurality of lane boundary detection points; determining coefficient values ​​of clothoid parameters for ground truth clothoid points and measured clothoid points formed using a ground truth set and a measured set, respectively, to model lane boundaries of a lane, wherein the ground truth set includes a subset of consecutive lane boundary detection points and the corresponding ground truth values, and the measured set includes a subset of consecutive lane boundary detection points and the corresponding measurement values; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane, the Kalman filter parameters being determined using at least one neural network; updating the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); In each cycle, minimizing a training error based on the difference between the reconstructed measured clothoid points and the corresponding ground truth set until the training error is below a predefined threshold. A lane tracking system (103) configured to: To determine the Kalman filter parameters using the at least one neural network, the processor (107) providing the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as inputs to a first neural network, the first neural network being trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using past measured sets and a ground truth set; using the first neural network to determine a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; predicting a state transition (Y p ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another based on the speed of the autonomous vehicle (101) moving along the lane and the time difference between successive image frames; determining a process noise covariance matrix (Q) using the predicted state transitions as inputs to a second neural network, the second neural network being trained using past autonomous vehicle (101) velocity and time difference values; predicting an error covariance (P p ) of the predicted state transition using the determined process noise covariance matrix (Q); determining the Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points based on the predicted state transitions (Y p ) and covariances (P p ), the determined measurement noise covariance matrix (R), and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; configured to: A lane tracking system (103).

16. 16. The lane tracking system (103) of claim 15, wherein the plurality of lane boundary detection points correspond to a left lane boundary and a right lane boundary of the lane.

17. 17. The lane tracking system of claim 15 or 16, wherein the processor dynamically selects the ground truth sets and the measured sets necessary to generate the ground truth clothoid points and the measured clothoid points, respectively.

18. 17. The lane tracking system (103) of claim 15 or 16, wherein the plurality of lane boundary detection points are determined using at least one of lane data received from one or more sensors (113) configured in the autonomous vehicle (101) and image frames of the lane received in real time, the image frames of the lane being received from an image capture device (115) associated with the autonomous vehicle (101).

19. 20. The lane tracking system (103) of claim 18, wherein the processor (107) obtains at least one of the lane data and the image frames of the lane from a database configured to store the lane data and the image frames captured in real time.

20. The clothoid parameters are the initial curvature of the lane boundary (c ｏ ), the rate of change of curvature of the lane boundary (c １ ) and a heading angle (β) relative to a direction of travel of the autonomous vehicle (101).

21. 17. The lane tracking system of claim 15, wherein the processor is further configured to add an initial lateral offset between the lane boundary and the autonomous vehicle to the reconstructed measured clothoid point.

22. 17. The lane tracking system (103) of claim 15 or 16, wherein the at least one neural network comprises a neural network with temporal memory.

23. 17. The lane tracking system (103) of claim 15 or 16, wherein the at least one neural network comprises a recurrent neural network.

24. 17. The lane tracking system (103) of claim 15 or 16, wherein the at least one neural network comprises a long short-term memory neural network.

25. A lane tracking system (103) for an autonomous vehicle (101), comprising: A processor (107); a memory (111) communicatively coupled to the processor (107), the memory (111) being configured, upon execution, to cause the processor (107) to: receiving measurements corresponding to a plurality of lane boundary detection points from a lane boundary detection system (105) associated with the lane tracking system (103); determining coefficient values ​​of clothoid parameters for measured clothoid points formed using a measurement set to model lane boundaries of a lane, the measurement set including a subset of consecutive lane boundary detection points and the corresponding measurements; determining Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points to track the lane boundaries of the lane, the Kalman filter parameters being determined using at least one neural network; updating the coefficient values ​​of the clothoid parameters determined for the measured clothoid points using the corresponding Kalman filter parameters; reconstructing the measured clothoid points using the corresponding updated coefficient values ​​of the clothoid parameters, each reconstructed measured clothoid point enabling the lane tracking system (103) to track the lane boundaries of the autonomous vehicle (101); a memory (111) storing instructions for causing a processor (107) to perform the A lane tracking system (103) comprising: To determine the Kalman filter parameters using the at least one neural network, the processor (107) providing the coefficient values ​​of the clothoid parameters determined for the measured clothoid points as inputs to a first neural network, the first neural network being trained based on past coefficient values ​​of the clothoid parameters determined for clothoid points formed using past measured sets and a ground truth set; using the first neural network to determine a measurement noise covariance matrix (R) using the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; predicting a state transition (Y p ) of the coefficient values ​​of the clothoid parameters determined for the measured clothoid points from one image frame to another based on the speed of the autonomous vehicle (101) moving along the lane and the time difference between successive image frames; determining a process noise covariance matrix (Q) using the predicted state transitions as inputs to a second neural network, the second neural network being trained using past autonomous vehicle (101) velocity and time difference values; predicting an error covariance (P p ) of the predicted state transition using the determined process noise covariance matrix (Q); determining the Kalman filter parameters for the coefficient values ​​of the clothoid parameters determined for the measured clothoid points based on the predicted state transitions (Y p ) and covariances (P p ), the determined measurement noise covariance matrix (R), and the coefficient values ​​of the clothoid parameters determined for the measured clothoid points; configured to: A lane tracking system (103).

26. 26. The lane tracking system (103) of claim 25, wherein the at least one neural network includes a neural network with temporal memory.

27. 27. The lane tracking system (103) of claim 25 or 26, wherein the at least one neural network comprises a recurrent neural network.

28. 27. The lane tracking system (103) of claim 25 or 26, wherein the at least one neural network comprises a long short-term memory neural network.

Images