Methods for training neural network-based motion controllers by simulating human driving behavior

Abstract

A variety of variations can include methods of training a neural network vehicle motion controller that more closely replicates how a human would drive a vehicle using intuitive vehicle dynamics variables and predict parameters in order to determine how a motion controller should communicate steering angle, throttle, and braking inputs to a vehicle to navigate the vehicle.

Description

Technical Field

[0001] This disclosure generally relates to the fields of vehicle motion controllers, and methods for making and using the same, including methods for simulating human driving behavior to train neural network-based vehicle motion controllers. Background Technology

[0002] Autonomous and semi-autonomous vehicles can use motion controllers to control the longitudinal and lateral movements of the vehicle. Summary of the Invention

[0003] Various variations may include vehicle motion controllers, as well as methods for making and using them (including methods for simulating human driving behavior to train neural network-based vehicle motion controllers).

[0004] Various variations may include methods for training a neural network vehicle motion controller that more closely replicates how a human would intuitively drive a vehicle using vehicle dynamics variables and predicts parameters to determine how the motion controller should transmit steering angle, throttle, and interrupt inputs to the vehicle for navigation.

[0005] Other illustrative variations within the scope of this invention will become apparent from the detailed description provided below. It should be understood that while disclosing variations within the scope of this invention, the detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of the invention. Attached Figure Description

[0006] Examples of variations within the scope of the invention will be more fully understood from the detailed description and accompanying drawings, wherein:

[0007] Figure 1 A method for training a neural network to simulate human driving behavior is shown, which may include characterizing the current state of the vehicle, what the driver sees in terms of path geometry, and the errors perceived by the driver through the application of steering and throttle / brake input corrections.

[0008] Figure 2 This is a block diagram of implementing a trained neural network, which includes trained parameters based on a neural network architecture, where X1 is... Figure 1 The vector shown is the training input vector, and Y1 is the vector of control parameters that are sent to the actuator to control the lateral and longitudinal motion of the vehicle.

[0009] Figure 3 This is a block diagram illustrating a method for training a neural network. Detailed Implementation

[0010] The following description of the variations is illustrative in nature and is in no way intended to limit the scope, application or use of the invention.

[0011] Various variations may include vehicle motion controllers, as well as methods for making and using them (including methods for simulating human driving behavior to train neural network-based vehicle motion controllers).

[0012] Various variations may include methods for training a neural network vehicle motion controller that more closely replicates how humans would drive a vehicle using "intuitive" feelings characterized by vehicle dynamics variables, and predicts parameters to determine how the motion controller should transmit steering angle, throttle, and interrupt inputs to the vehicle for navigation.

[0013] Prior to this, lateral and longitudinal vehicle motion controllers were separate entities, and their effects on vehicle dynamics were inferred only when control inputs were provided to the vehicle actuators. These types of motion control methods resulted in them exhibiting very robotic or unnatural vehicle behavior, which felt distinctly unfamiliar and uncomfortable to human vehicle drivers and / or passengers.

[0014] In various variations, the predicted data can be used or parameterized as a system of equations represented by multi-order differential equations. This data can then be fed into a neural network in an input-output format (pre-prepared so that the network receives weights and biases that will fit the input data set as closely as possible). These weights and biases can then be deployed as a "homogeneous motion controller" to achieve lateral and longitudinal vehicle motion control in autonomous or semi-autonomous mode. The same applies to braking. The weights and biases can be deployed as a "homogeneous motion controller" to achieve vehicle deceleration motion control in autonomous or semi-autonomous mode. The inputs to such a vehicle motion controller will be exactly the same in terms of variables as those used during training. However, due to the general nature of neural networks, when compared with the training data, the neural network will behave like a robot in response to changes and will be able to drive forward on the desired road at the desired rate required by the path planner. Because the neural network has already been trained on the same input vector based on the learned behavior simulated in terms of weights, biases, and associated process uncertainties, the controller's output will closely match what a human would do (if presented to them with the same set of inputs). This will allow the vehicle to traverse the path in a human-like manner, even if the controller itself is not human.

[0015] In various variants, the uniform motion controller can provide lateral and longitudinal motion control signals that mimic human driving behavior. In various variants, the uniform motion controller can be constructed and arranged to provide individual and distinct driving behavior characteristics by training the neural network with human vehicle drivers possessing different driving personalities or characteristics. In various variants, the uniform motion controller can have the ability to continuously learn driver behavior and adapt it using weights and biases, and to update the neural network from time to time. The neural network can be trained to drive the vehicle with a variety of different personalities or characteristics, such as an aggressive first driving characteristic, where the driver turns quickly or sharply and accelerates and / or decelerates aggressively or quickly; a gentler second driving characteristic, where the driver turns gently or less sharply and accelerates and / or decelerates gently or more slowly than the first driving characteristic; and a third driving characteristic, which is more conservative than the second driving characteristic, where the driver turns more slowly and less sharply and accelerates and decelerates more slowly or conservatively than the second driving characteristic. The trained neural network will be constrained downstream to keep it within safe operating limits for the vehicle and environment, regardless of the learned behavior.

[0016] refer to Figure 1 The current state of the vehicle relative to a plurality of variables can be determined using vehicle 10 (which may include multiple sensors 12, 14 and one or more modules or computing devices 15), including at least one of yaw 18, speed 20, lateral acceleration 22, longitudinal acceleration 24, yaw rate 26, steering wheel rate 28, steering wheel angle 30, or steering angle target 32. The current state of the vehicle relative to these parameters can be recorded at multiple time points (such as t=0 and t=1 when vehicle 10 is moving along path 11). The neural network can also record the driver's predictions 34 relative to a plurality of variables, including at least one of X direction 36, Y direction 38, coefficient #1 40, coefficient #2 42, and coefficient #3 44, wherein coefficients #1, #2, and #3 represent a feature or parametric curve equation, lateral deviation of the vehicle from the intended path 46, deviation of the vehicle's current heading from the intended path heading 48, curvature of the future trajectory 50, or target speed 52. One or more of these variables can be obtained through one or more modules or computing devices 15. Other parameters, such as environmental conditions, road surface friction, and vehicle health information, can be added to the current vehicle status 16.

[0017] Now for reference Figure 2Input data can be fed into the neural network, where such input data comes from the current state of vehicle 16 and the driver's prediction 34, as well as other parameters required, such as whether an output is needed for an aggressive first driving characteristic, a mild second driving characteristic, or a conservative third driving characteristic. The neural network will be a separate controller and can work independently or in conjunction with existing conventional control functions, and can compare or average the outputs of each.

[0018] Various variations may include methods for training neural networks, which involve having a human driver drive on a test track at a first rate for a first driving characteristic and using multiple sensors 12, 14 and one or more modules or computing devices 15 to determine the current state of the vehicle at multiple points in time using at least one of yaw 18, speed 20, lateral acceleration 22, longitudinal acceleration 24, yaw rate 2, 6, speed 28, steering wheel angle 30 or steering angle target 32, and to determine the predictions made by the driver, i.e., X direction 36, Y direction 38, coefficient #1 40, coefficient #2 42, coefficient #3 At least one of 44, wherein coefficients #1, #2, and #3 represent a feature or parametric curve equation, a lateral deviation 46 of the vehicle from the intended path, a directional deviation 48 of the vehicle's current heading from the intended path, a curvature 50 of the future trajectory, or a target speed 52, and, based on the determination, generating input data, and transmitting the input data to a neural network to simulate human driving behavior and generate output data therefrom, and transmitting the output data to an autonomous driving vehicle module constructed and arranged to drive the vehicle for at least a period of time without human input. The first rate may be at a relatively fast speed to simulate the human driving behavior of an aggressive driver. The same process may be repeated at a second rate lower than the first rate to simulate the human driving behavior of a moderate driver. Similarly, for a third rate lower than the second rate, the same process may be repeated to simulate the human driving behavior of a conservative driver.

[0019] Various variations may include a trained neural network constructed and arranged to produce output data, which has been trained by receiving input data obtained in such a way as to: having a human driver drive on a test track at a first rate for a first driving characteristic and using multiple sensors 12, 14 and one or more modules or computing devices 15 to determine the current state of the vehicle at multiple points in time using at least one of yaw 18, speed 20, lateral acceleration 22, longitudinal acceleration 24, yaw rate 2, 6, speed 28, steering wheel angle 30 or steering angle target 32, and to determine the prediction made by the driver, namely at least one of X direction 36, Y direction 38, coefficient #1 40, coefficient #2 42, coefficient #3 44, wherein coefficients #1, #2 and #3 represent a characteristic or parametric curve equation, lateral deviation of the vehicle from the intended path 46, deviation of the vehicle's current heading from the intended path heading 48, curvature of the future trajectory 50 or target speed 52.

[0020] In addition to the training methods described above, once the vehicle has been delivered to the customer with a basically trained neural network, the software module can be activated to continuously record vehicle status, prediction information, and driver input when the driver manually operates the vehicle. If the recorded information is determined to originate from driving feature regions already considered to have low reliability in the trained neural network, this information will be fed back into the neural network as supplementary information, and the weights, biases, and uncertainties will be updated. This process ensures continuous learning and improvement of the neural network uniform controller.

[0021] Now for reference Figure 3 Various variations may include methods for training neural networks, which include initial neural network training and development behaviors, the initial neural network training and development behaviors including: collecting data such as... Figure 1The process involves: 302, processing actual driving data from multiple drivers at a given set of comfort parameters and speeds; preprocessing the driving data to feed it into a training algorithm; 304, training a multi-level deep network using a neural network / machine learning training algorithm, in which various uncertainties and data mean and standard deviation are understood, and the set of weights and biases are used as mathematical expressions for the human driver's response to a given set of inputs; 306, using the weights and biases to generate lateral and longitudinal motion controllers for controlling the vehicle's trajectory; and then performing ongoing or subsequent neural network training and development activities, including: collecting data while the human driver continues driving in manual mode once the trained neural network is deployed; 310, uploading the data to cloud infrastructure or computing resources on the vehicle, evaluating the neural network against the new training data at the cloud infrastructure or computing resources on the vehicle, and comparing the uncertainties, mean, and biases with the original trained neural network; and 314, updating the weights and biases if the differences are considered to improve the performance of the neural network and are within safety limits, and if acceptable to the vehicle owner / driver.

[0022] The above description of the alternative variations within the scope of the invention is illustrative in nature only, and therefore such variations or variants shall not be considered as departing from the spirit and scope of the invention.

Claims

1. A method for training a neural network, comprising having a human driver drive a vehicle on a test track at a first rate for a first driving characteristic and using multiple sensors and one or more modules or computing devices to determine the current state of the vehicle at multiple time points using at least one of yaw, speed, lateral acceleration, longitudinal acceleration, yaw rate, rate, steering wheel angle, and steering angle targets, wherein, For the first driving characteristic, the human driver turns quickly or sharply, and accelerates or decelerates aggressively or rapidly; And determine the driver’s prediction, including at least one of the following: path geometry in the X and Y directions in front of the vehicle, lateral deviation of the vehicle from the intended path, deviation of the vehicle’s current heading from the intended path, curvature of the future trajectory, and target speed; generate input data based on the above determination; transmit the input data to a neural network to simulate human driving behavior and generate output data from the neural network; and transmit the output data to an autonomous driving vehicle module constructed and arranged to drive the vehicle for at least a period of time without human input. The method further includes having a human driver drive the vehicle on a test track at a second rate for a second driving characteristic and using multiple sensors and one or more modules or computing devices to determine the current state of the vehicle at multiple time points using at least one of yaw, speed, lateral acceleration, longitudinal acceleration, yaw rate, rate, steering wheel angle, and steering angle targets, wherein, for the second driving characteristic, the human driver turns less sharply and accelerates or decelerates more slowly than in the first driving characteristic; and determining a prediction made by the driver, including at least one of the following: path geometry in the X and Y directions ahead of the vehicle, lateral deviation of the vehicle from the intended path, deviation of the vehicle's current heading from the intended path, curvature of the future trajectory, and target speed; generating input data based on the above determination, and transmitting the input data to a neural network to simulate human driving behavior and generating output data from the neural network, and transmitting the output data to an autonomous driving vehicle module constructed and arranged to drive the vehicle without human input for at least a period of time, wherein the second rate is less than the first rate. The method further includes having a human driver drive the vehicle on a test track at a third rate for a third driving characteristic and using multiple sensors and one or more modules or computing devices to determine the current state of the vehicle at multiple time points using at least one of yaw, speed, lateral acceleration, longitudinal acceleration, yaw rate, rate, steering wheel angle, and steering angle targets, wherein, for the third driving characteristic, the human driver turns more slowly and less sharply, and accelerates or decelerates more slowly than in the second driving characteristic; and determining a prediction made by the driver, including at least one of path geometry in the X and Y directions ahead of the vehicle, lateral deviation of the vehicle from the intended path, deviation of the vehicle's current heading from the intended path, curvature of the future trajectory, and target speed; generating input data based on the above determination, and transmitting the input data to a neural network to simulate human driving behavior and generating output data from the neural network, and transmitting the output data to an autonomous driving vehicle module constructed and arranged to drive the vehicle without human input for at least a period of time, wherein the third rate is less than the second rate. The trained neural network is constrained downstream to keep it within safe operating limits.

2. A system for constructing and arranging a trained neural network to generate output data, wherein the neural network has been trained by the method of claim 1.

3. A method comprising training a system according to claim 2 having a predetermined neural network model architecture, the method comprising: Before feeding the grouped training data into the neural network, the inherent uncertainty of the grouped training data and the uncertainty within the predetermined neural network model architecture are determined, resulting in data preprocessing to determine homoscedasticity and homoscedastic uncertainty, and these are used as inputs to allow the neural network to understand and learn how the inputs are distributed in the driving space, as well as to learn / adjust the mean and standard deviation associated with each network neuron of the neural network weights and biases.

Images