Methods for modeling human driving behavior for training motion controls based on a neural network

Abstract

Methods for training a neural network comprising: • Driving by a human driver on a test track at an initial speed for initial driving characteristics and using a variety of sensors (12, 14) and one or more modules or computing devices (15) which determine the current state (16) of the vehicle (10) at different times using a yaw (18) and / or a speed (20) and / or a lateral acceleration (22) and / or a longitudinal acceleration (24) and / or a yaw rate (26) and / or a steering wheel speed (28) and / or a steering wheel angle (30) and / or a steering angle target (32); and determine what the driver sees ahead (34) with respect to the X-direction (36), the Y-direction (38), a coefficient #1 (40), a coefficient #2 (42), a coefficient #3 (44), where the coefficients #1 (40), #2 (42),and #3 (44) represent the characteristic or parametric curve equation, the lateral deviation (46) of the vehicle (10) from an intended path (11), a direction deviation (48) of the current direction of travel of the vehicle (10) from the intended path (11), a curvature of the future trajectory (50), and a target speed (52), and generating input data (54) from the determination, and communicating the input data (54) to a neural network (56, 58) to model human driving behavior, and generating output data (60) from the neural network (56, 58), and communicating the output data (60) to a module of an autonomously driving vehicle (10) which is designed and configured to drive a vehicle (10) at least for a period of time without human input,further comprehensive: • Driving on a test track by a human driver at a second speed for a second driving characteristic and using a variety of sensors (12, 14) and one or more modules or computing devices (15) which determine the current state (16) of the vehicle at different times using a yaw (18) and / or a speed (20) and / or a lateral acceleration (22) and / or a longitudinal acceleration (24) and / or a yaw rate (26) and / or a steering wheel speed (28) and / or a steering wheel angle (30) and / or a steering angle target (32); and• Determine what the driver sees ahead (34) with respect to the X-direction (36), the Y-direction (38), a coefficient #1 (40), a coefficient #2 (42), a coefficient #3 (44), where the coefficients #1 (40), #2 (42), and #3 (44) represent the characteristic or parametric curve equation,a lateral deviation (46) of the vehicle (10) from an intended path (11), a direction deviation (48) of the current direction of travel of the vehicle (10) from an intended path (11), a curvature of the future trajectory (50), and a target speed (52), and generating input data (54) from the determination, and communicating the input data (54) to a neural network (56, 58) to model human driving behavior and to generate output data (60) from the neural network (56, 58), and communicating the output data (60) to a module of an autonomously driving vehicle (10) which is designed and configured to drive the vehicle (10) for at least a period of time without human input, and wherein the second speed is lower than the first speed.

Description

Technical field

[0001] The area to which the disclosure generally refers includes vehicle motion controls and methods for manufacturing and using the same, comprising a method for modeling human driving behavior for training vehicle motion controls based on neural networks. background

[0002] Autonomous and semi-autonomous vehicles can use motion controllers to control longitudinal and lateral movement of the vehicle. Summary of illustrative variations

[0003] Some illustrative variations may include vehicle motion controllers and methods for manufacturing and using them, including a method for modeling human driving behavior for training vehicle motion controllers based on neural networks.

[0004] Some variations may include a method for training a neural network-based vehicle motion control system that more accurately replicates how a human would drive a vehicle, using intuitive vehicle dynamics variables and anticipatory parameters to determine how a motion control system will manage the steering angle, accelerator pedal and brake inputs for the vehicle to navigate the vehicle.

[0005] Further illustrative variations within the scope of the invention will become clearer with reference to the detailed description provided below. It should be understood that while the detailed description and specific examples disclose variations within the scope of the invention, they are intended solely for illustrative purposes and are not intended to limit the scope of the invention. List of characters

[0006] Selected examples of variations within the scope of protection of the invention become clearer with reference to the detailed description and the accompanying figures, wherein: Fig. 1. A method for training a neural network to model human driving behavior is illustrated, which may include characterizing the current state of the vehicle and what the driver anticipates regarding path geometry and perceived errors, which the driver corrects by applying steering and accelerator / brake input. Fig. 2 is a block diagram of an implementation of the trained neural network, which includes trained parameters based on the neural network architecture, where X1 is a vector of training inputs that are in Fig. Figure 1 shows Y1 as a vector of control parameters which are sent to the actuators to control the lateral and longitudinal movement of the vehicle. Fig. 3 is a block diagram that illustrates a method for training a neural network. Detailed description of illustrative variations

[0007] The following description of variations is purely illustrative and is in no way intended to limit the scope of protection of the invention, its application or uses.

[0008] Some variations may include vehicle motion controls and methods for manufacturing and using them, including a method for modeling human driving behavior to train vehicle motion controls based on a neural network.

[0009] Some variations may include a method for training a neural network-based vehicle motion control system that more accurately replicates how a human would drive a vehicle, using an "intuitive" feel characterized by vehicle dynamics variables and anticipatory parameters to determine how a motion control system will manage the steering angle, accelerator pedal and brake inputs for the vehicle to navigate the vehicle.

[0010] Previously, lateral and longitudinal vehicle motion control systems were separate and only inferred their mutual influences on vehicle dynamics when control inputs were provided to the vehicle actuators. These types of motion control methods result in very robotic and unnatural vehicle behavior, which feels distinctly unusual and unpleasant for a human driver and / or passenger.

[0011] In some variations, pre-existing data, as presented here, can be used, or it can be parameterized to a set of equations represented by higher-order differential equations. This data can then be fed into a neural network in the previously prepared input / output format to obtain a network with weights and thresholds (biases) that closely matches the input dataset. These weights and thresholds can subsequently be used as a homogeneous motion control mechanism to achieve lateral and longitudinal vehicle motion control in autonomous or semi-autonomous modes. The same can be achieved for braking. Weights and thresholds can be used as a homogeneous motion control mechanism to achieve motion control of vehicle deceleration in autonomous or semi-autonomous modes.The inputs for such a vehicle motion control system are exactly the same as those used during training with respect to the variables. However, due to the generalizing nature of the neural network, it will behave robotically with respect to deviations from the training data and will be able to travel along the road ahead at a desired speed required by a path planner. Because the neural network was trained using the same vector of inputs, the control output, based on the learned behavior modeled by the weights, thresholds, and associated process uncertainties, will closely match what a human would have done given the same set of inputs. This allows the vehicle to navigate the path in a human-like manner, even if the control itself is not performed by a human.

[0012] In some variations, a homogeneous motion controller can provide lateral and longitudinal motion control signals that mimic human driving behavior. In some variations, the homogeneous motion controller can be designed and configured to provide personalities and varying driving behavior characteristics by training the neural network with human drivers of different personalities and character traits. In some variations, the homogeneous motion controller can have the ability to continuously learn and adapt to the driver's behavior using weights and thresholds, and to update the neural network periodically.The neural network can be trained by driving the vehicle with a variety of different personalities and characteristics, such as a first driving characteristic that is aggressive, in which the driver changes direction quickly or sharply, and accelerates and / or decelerates aggressively or quickly; a second driving characteristic that is more moderate than the first driving characteristic, in which the driver changes direction in a moderate or less sharp manner, and accelerates and decelerates in a moderate or less rapid manner than in the first driving characteristic; and a third driving characteristic that is more conservative than the second driving characteristic, in which the driver changes direction more slowly and less sharply, and accelerates or decelerates in a slower or more conservative manner than in the second driving characteristic.The trained neural network is subsequently restricted to remain within the limits of safe operation of the vehicle and its environment, regardless of any learned behavior.

[0013] Referring to Fig. 1. A vehicle 10 can have a variety of sensors 12, 14, and one or more modules or computing devices 15 can be used to determine the current state of a vehicle with respect to a variety of variables, including yaw 18, velocity 20, lateral acceleration 22, longitudinal acceleration 24, yaw rate 26, steering wheel speed 28, steering wheel angle 30, or steering angle target 32. The current state of a vehicle with respect to these parameters can be recorded at different times, such as t=0 and t=1, while the vehicle 10 is moving along a path 11.Furthermore, the neural network can record what the driver sees ahead 34 with respect to a variety of variables, which include the X-direction 36 and / or the Y-direction 38 and / or a coefficient #1 40 and / or a coefficient #2 42 and / or a coefficient #3 44, where coefficients #1, #2, and #3 represent a characteristic or parametric curve equation, a lateral deviation of the vehicle from the intended path 46, a directional deviation of the vehicle from the direction of travel of the intended path 48, a curvature of the future trajectory 50, or a target speed 52. One or more of these variables can be obtained by one or more modules or computing devices 15. Additional parameters, such as environmental conditions, road friction, and vehicle health information, can be added to the current vehicle state 16.

[0014] Referring to Fig. 2. The input data can be transmitted to a neural network, where such input data is derived from the current state of the vehicle 16 and what a driver sees ahead 34, and, if necessary, from further parameters such as whether the output is needed for the first driving characteristic, which is aggressive, for the second driving characteristic, which is moderate, or for the third driving characteristic, which is conservative. The neural network would be a separate controller and can either operate independently or be linked to existing conventional control functions, and their respective outputs can be compared or averaged.

[0015] Some variations may include a procedure for training a neural network, which comprises: driving by a human driver on a test track at an initial speed for initial driving characteristics and using a variety of sensors 12, 14 and one or more modules or computing devices 15, determining the current state of the vehicle at different times using a yaw rate 18 and / or a speed 20, and / or a lateral acceleration 22 and / or a longitudinal acceleration 24 and / or a yaw rate 26 and / or a steering wheel speed 28 and / or a steering wheel angle 30 and / or a steering angle target 32, and determining what the driver sees ahead with respect to the X-direction 36 and / or the Y-direction 38 and / or a coefficient #1 40 and / or a coefficient #2 42 and / or a coefficient #3 44, where the coefficients #1, #2,and #3 represent the characteristic or parametric curve equation and / or a lateral deviation of the vehicle from the intended path 46 and / or a directional deviation of the vehicle from a direction of travel of an intended path 48 and / or a curvature of the future trajectory 50 and / or a target speed 52, and generating input data from the determination, and communicating the input data to a neural network to model human driving behavior, and generating output data based on this, and communicating the output data to a module of an autonomous vehicle, which is designed and configured to drive a vehicle for at least a period of time without human input. The first speed can be a relatively high speed to model the human driving behavior of an aggressive driver. The same process can be carried out with a second speed,The process can be repeated for a third speed, lower than the first, to model the driving behavior of a moderate driver. Similarly, the same process can be repeated for a third speed, lower than the second, to model the driving behavior of a conservative driver.

[0016] Some variations may include a trained neural network which is constructed and configured to generate output data, wherein the neural network was trained by receiving input data obtained by a human driver driving on a test track at an initial speed for an initial driving characteristic and by using a variety of sensors 12, 14 and one or more modules or computing devices 15 to determine the current state of the vehicle at different times, using a yaw rate 18 and / or a speed 20 and / or a lateral acceleration and / or a longitudinal acceleration 24 and / or a yaw rate 26 and / or a steering wheel speed 28 and / or a steering wheel angle 30 and / or a steering angle target 32, and by determiningwhat the driver sees ahead with respect to the X-direction 36 and / or the Y-direction 38 and / or a coefficient #1 40 and / or a coefficient #2 42 and / or a coefficient #3 44, where the coefficients #1, #2, and #3 represent the characteristic or parametric curve equation, and / or a lateral deviation of the vehicle from an intended path 46 and / or a directional deviation of a current direction of travel of the vehicle from the intended path 48 and / or a curvature of the future trajectory 50 and / or the target speed 52.

[0017] In addition to the training procedure described above, once the vehicle has been delivered to the customer with a basic trained neural network, a software module can be executed to continuously record the vehicle state, anticipatory information, and driver inputs in cases where the driver operates the vehicle manually. If it is determined that the recorded information originates from a region of a driving characteristic considered to have lower confidence in the trained neural network, the information is fed back into the neural network as additional data, and the weights, thresholds, and uncertainties are updated. This process ensures continuous learning and continuous improvement of the homogeneous control system based on the neural network.

[0018] Referring to Fig. 3. Some variations may include a procedure for training a neural network, which comprises: initial training of the neural network and developmental processes, which, as by means of Fig.1 described, including the acquisition 302 of actual driving data for multiple drivers driving with a predefined set of comfort parameters and speeds; preprocessing 304 of the driving data to enable its input into a training algorithm; using 306 a neural network / machine learning training algorithm to train a multi-layered deep network in which the different uncertainties along with means and standard deviations of data are understood and in which this set of weights and thresholds is used as a mathematical representation of a human driver's response to a predefined set of inputs; using 308 the weights and thresholds to generate a lateral and longitudinal motion control that governs the vehicle's trajectory;and subsequent execution of ongoing or downstream training of the neural network and development processes, which include the acquisition of data while the human driver continues to drive in a manual mode once the trained neural network is deployed; uploading the data either to the cloud infrastructure or to an onboard computing resource, where the neural network is evaluated with respect to new training data and where uncertainties, means, and thresholds are compared with the originally trained neural network; and, if differences are considered to improve the performance of the neural network within the safety limits, updating the weights and thresholds if this is acceptable to the owner / driver of the vehicle.

[0019] The above description of selected variations within the scope of protection of the invention is purely illustrative and consequently variations or variants thereof are not to be regarded as a deviation from the spirit and scope of protection of the invention. Reference symbol list 10 vehicles Path 11 12 Sensor 14 Sensor 15 computing device 16 Vehicle condition 18 Greed 20 speed 22 Lateral acceleration 24 Longitudinal acceleration 26 Yaw rate 28 Steering wheel speed 30 steering wheel angle 32 Steering angle target 34 what the driver sees ahead 36 X-direction 38 Y-direction 40 Coefficient #1 42 Coefficient #2 44 Coefficient #3 46 Lateral deviation from the intended path 48 Deviation from the intended path 50 future trajectories 52 Target speed 54 Vector of training inputs X1 56 Input of the function-adapting neural network 58 Output of the function-adapting neural network 60 Vector of control parameters Y1 302 - 316 Procedural steps

Claims

[1] Methods for training a neural network including: • Driving by a human driver on a test track at an initial speed for initial driving characteristics and using a variety of sensors (12, 14) and one or more modules or computing devices (15) which determine the current state (16) of the vehicle (10) at different times using a yaw rate (18) and / or a speed (20) and / or a lateral acceleration (22) and / or a longitudinal acceleration (24) and / or a yaw rate (26) and / or a steering wheel speed (28) and / or a steering wheel angle (30) and / or a steering angle target (32); and • Determine what the driver sees ahead (34) with respect to the X-direction (36) and / or the Y-direction (38) and / or a coefficient #1 (40) and / or a coefficient #2 (42) and / or a coefficient #3 (44), where the coefficients #1 (40), #2 (42), and #3 (44) represent the characteristic or parametric curve equation, and / or the lateral deviation (46) of the vehicle (10) from an intended path (11) and / or a direction deviation (48) of the current direction of travel of the vehicle (10) from the intended path (11) and / or a curvature of the future trajectory (50) and / or a target speed (52), and generate input data (54) from the determination, and communicate the input data (54) to a neural network (56, 58) to simulate human driving behavior modeling and generating output data (60) from the neural network (56, 58),and communicating the output data (60) to a module of an autonomously driving vehicle (10) which is designed and equipped to drive a vehicle (10) for at least a period of time without human input. [2] The method according to claim 1 further comprising: • Driving on a test track by a human driver at an initial speed for initial driving characteristics and using a variety of sensors (12, 14) and one or more modules or computing devices (15) which determine the current state (16) of the vehicle at different times using a yaw rate (18) and / or a speed (20) and / or a lateral acceleration (22) and / or a longitudinal acceleration (24) and / or a yaw rate (26) and / or a steering wheel speed (28) and / or a steering wheel angle (30) and / or a steering angle target (32); and • Determine what the driver sees ahead (34) with respect to the X-direction (36) and / or the Y-direction (38) and / or a coefficient #1 (40) and / or a coefficient #2 (42) and / or a coefficient #3 (44), where the coefficients #1 (40), #2 (42), and #3 (44) represent the characteristic or parametric curve equation, and / or a lateral deviation (46) of the vehicle (10) from an intended path (11) and / or a direction deviation (48) of the current direction of travel of the vehicle (10) from an intended path (11) and / or a curvature of the future trajectory (50) and / or a target speed (52), and generate input data (54) from the determination, and communicate the input data (54) to a neural network (56, 58) to simulate human driving behavior model and to generate output data (60) from the neural network (56, 58),and communicating the output data (60) to a module of an autonomously driving vehicle (10) which is designed and configured to drive the vehicle (10) for at least a period of time without human input, and wherein the second speed is lower than the first speed. [3] Method according to claim 1 or 2 further comprising: • Driving on a test track by a human driver at an initial speed for initial driving characteristics and using a variety of sensors (12, 14) and one or more modules or computing devices (15) which determine the current state (16) of the vehicle at different times using a yaw rate (18) and / or a speed (20) and / or a lateral acceleration (22) and / or a longitudinal acceleration (24) and / or a yaw rate (26) and / or a steering wheel speed (28) and / or a steering wheel angle (30) and / or a steering angle target (32); and • Determine what the driver sees ahead (34) with respect to the X-direction (36) and / or the Y-direction (38) and / or a coefficient #1 (40) and / or a coefficient #2 (42) and / or a coefficient #3 (44), where the coefficients #1 (40), #2 (42), and #3 (44) represent the characteristic or parametric curve equation, and / or a lateral deviation (46) of the vehicle (10) from the intended path (11) and / or a direction deviation (28) of the current direction of travel of the vehicle (10) from the intended path (11) and / or a curvature of the future trajectory (50) and / or a target speed (52), and generate input data (54) from the determination, and communicate the input data (54) to a neural network (56, 58) to model human driving behavior and to generate output data (60) from the neural network (56, 58),and communicating the output data (60) to a module of an autonomously driving vehicle (10) which is designed and equipped to drive a vehicle (10) for at least a period of time without human input, and wherein the third speed is lower than the second speed. [4] Trained neural network (56, 58) which is constructed and set up to generate output data (60), wherein the neural network (56, 58) has been trained by receiving input data (54) obtained by a human driver driving on a test track at an initial speed for an initial driving characteristic and by using a variety of sensors (12, 14), and one or more modules or computing devices (15) by determining the current state (16) of the vehicle (10) at different times, using a yaw (18) and / or a speed (20) and / or a lateral acceleration (22) and / or a longitudinal acceleration (24) and / or a yaw rate (26) and / or a steering wheel speed (28) and / or a steering wheel angle (30) and / or a steering angle target (32), and by determining,what the driver sees with respect to the X-direction (36) and / or the Y-direction (38) and / or a coefficient #1 (40) and / or a coefficient #2 (42) and / or a coefficient #3 (44), wherein the coefficients #1 (40), #2 (42), and #3 (44) represent the characteristic or parametric curve equation, and / or a lateral deviation (46) of the vehicle (10) from the intended path (11) and / or a direction deviation (48) of the current direction of travel of the vehicle (10) from the intended path (11) and / or the curvature of the future trajectory (50) and / or a target speed (52), and by generating input data (54) from the determination, and by feeding the input data (54) into a neural network (56, 58) to model human driving behavior. [5] Method comprising training a neural network (56, 58) which has a predefined neural network model architecture, wherein the method includes: determining the inherent uncertainties within a training data set and the uncertainties within the neural network model architecture prior to feeding the training data set, which causes homoscedastic and heteroscedastic uncertainties to be determined in data preprocessing and using these as inputs to enable the neural network (56, 58) to understand and learn how the inputs are distributed within the driving space and to learn / adjust the mean and standard deviations which are associated with each network neuron of the neural weights and thresholds.

Images