Driver command prediction

Abstract

A driver command predictor includes a controller, a plurality of sensors, and a command prediction unit. The controller is configured to command adjustments to a plurality of motion vectors of a vehicle relative to a roadway in response to a plurality of actual driver commands and a plurality of future driver commands. The actual driver commands are received at a current time. The future driver commands are received at a plurality of update times. The update times range from the current time to a future time. The sensors are configured to generate sensor data that determines a plurality of actual states of the vehicle in response to the motion vectors as commanded. The command prediction unit is configured to generate the future driver commands at the update times in response to a driver model. The driver model operates on the actual driver commands and the actual states to predict the future driver commands at the update times.

Description

Technical Field

[0001] This disclosure relates to systems and methods for predicting driver commands. Background Technology

[0002] When developing model predictive controllers for vehicle motion control, it is assumed that driver input commands remain constant within a finite range, predicting the vehicle's state within that same range. This assumption may be inaccurate because drivers sometimes react to dynamic road conditions. Therefore, this assumption can lead to suboptimal predictive control actions.

[0003] The desired technology is one for predicting driver commands. Summary of the Invention

[0004] This document provides a driver command predictor. The driver command predictor includes a controller, multiple sensors, and a command prediction unit. The controller is configured to command adjustments to one or more motion vectors of a vehicle relative to a road in response to multiple actual driver commands and multiple future driver commands. Multiple actual driver commands are received at the current time. Multiple future driver commands are received at multiple update times. The multiple update times range from the current time to future times. The multiple sensors are configured to generate multiple sensor data determining multiple actual states of the vehicle in response to the commanded one or more motion vectors. The command prediction unit is configured to generate the multiple future driver commands in response to a driver model at the multiple update times. The driver model operates on the multiple actual driver commands and the multiple actual states to predict the multiple future driver commands at the multiple update times.

[0005] In one or more embodiments, the driver command predictor includes one or more information means configured to generate road information. The driver model operates on the road information to generate multiple future driver commands at multiple update times.

[0006] In one or more embodiments of the driver command predictor, the driver model is a deep learning model.

[0007] In one or more embodiments of the driver command predictor, a deep learning model is configured to generate multiple desired vehicle states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

[0008] In one or more embodiments of the driver command predictor, the command prediction unit generates a plurality of future driver commands to make the plurality of desired states conform to the plurality of actual states.

[0009] In one or more embodiments of the driver command predictor, the command prediction unit is configured to generate multiple results by comparing the plurality of future driver commands with the plurality of actual driver commands, and update the driver model based on the plurality of results.

[0010] In one or more embodiments of the driver command predictor, the controller treats the plurality of actual driver commands as a plurality of constants between the current time and future time.

[0011] In one or more embodiments of the driver command predictor, the controller includes a driver command interpreter and a model prediction controller. The driver command interpreter is configured to generate a plurality of intermediate commands in response to the plurality of actual driver commands and a plurality of future driver commands. The model prediction controller commands one or more motion vectors of the vehicle to be adjusted at a plurality of update times in response to the plurality of intermediate commands and the plurality of future driver commands.

[0012] In one or more embodiments of the driver command predictor, the future time is in the range of 100 to 500 milliseconds after the current time.

[0013] This paper provides a method for predicting driver commands. The method includes adjusting one or more motion vectors of a vehicle relative to a road in response to multiple actual driver commands and multiple future driver commands. Circuitry receives multiple actual driver commands at a current time. The method also includes generating multiple sensor data determining multiple actual states of the vehicle in response to the commanded one or more motion vectors, and generating multiple future driver commands at multiple update times in response to a driver model. The multiple update times range from the current time to future times. The driver model operates on the multiple actual driver commands and the multiple actual states to predict the multiple future driver commands at the multiple update times.

[0014] In one or more embodiments, the method includes generating road information. A driver model operates on the road information to generate multiple future driver commands at multiple update times.

[0015] In one or more embodiments of the method, the driver model is a deep learning model.

[0016] In one or more embodiments of the method, a deep learning model is configured to generate multiple desired vehicle states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

[0017] In one or more embodiments of the method, a plurality of future driver commands are generated to make the plurality of desired states conform to the plurality of actual states.

[0018] In one or more embodiments, the method includes generating a plurality of results by comparing the plurality of future driver commands with the plurality of actual driver commands, and updating a driver model based on the plurality of results.

[0019] In one or more embodiments of the method, the plurality of actual driver commands are regarded as a plurality of constants between the current time and future time.

[0020] This document provides a vehicle. The vehicle includes multiple driver controllers and a driver command predictor. The multiple driver controllers are configured to generate multiple actual driver commands. The driver command predictor is configured to command adjustments to one or more motion vectors of the vehicle relative to a road in response to the multiple actual driver commands and multiple future driver commands. Multiple actual driver commands are received at the current time. The driver command predictor is also configured to generate multiple sensor data determining multiple actual states of the vehicle in response to the commanded one or more motion vectors, and to generate multiple future driver commands at multiple update times in response to a driver model. The multiple update times range from the current time to future times. The driver model operates on the multiple actual driver commands and the multiple actual states to predict the multiple future driver commands at the multiple update times.

[0021] In one or more embodiments of the vehicle, the driver command predictor includes one or more information devices configured to generate road information, wherein the driver model operates on the road information to generate multiple future driver commands at multiple update times.

[0022] In one or more embodiments of the vehicle, the driver model is a deep learning model, and the deep learning model is configured to generate multiple desired vehicle states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

[0023] In one or more embodiments of the vehicle, a driver command predictor generates multiple future driver commands to make the multiple desired states conform to multiple actual states.

[0024] This invention provides the following technical solutions:

[0025] 1. A driver command predictor, comprising:

[0026] A controller is configured to command the adjustment of one or more motion vectors of the vehicle relative to a road in response to a plurality of actual driver commands and a plurality of future driver commands, wherein the plurality of actual driver commands are received at the current time, the plurality of future driver commands are received at a plurality of update times, and the plurality of update times range from the current time to future times.

[0027] Multiple sensors, configured to generate multiple sensor data to determine multiple actual states of the vehicle in response to one or more motion vectors as commanded; and

[0028] A command prediction unit is configured to generate the plurality of future driver commands in response to a driver model at multiple update times, wherein the driver model operates on the plurality of actual driver commands and the plurality of actual states to predict the plurality of future driver commands at the plurality of update times.

[0029] The driver command predictor according to technical solution 1 further includes:

[0030] One or more information devices are configured to generate road information, wherein the driver model operates on the road information to generate the plurality of future driver commands at the plurality of update times.

[0031] According to the driver command predictor of technical solution 1, the driver model is a deep learning model.

[0032] According to the driver command predictor of technical solution 3, the deep learning model is configured to generate multiple desired vehicle states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

[0033] According to the driver command predictor of technical solution 4, the command prediction unit generates multiple future driver commands so that the multiple expected states conform to the multiple actual states.

[0034] According to the driver command predictor of technical solution 1, the command prediction unit is further configured to:

[0035] Multiple results are generated by comparing the multiple future driver commands with the multiple actual driver commands; and

[0036] The driver model is updated based on the multiple results.

[0037] According to the driver command predictor of technical solution 1, the controller treats the plurality of actual driver commands as a plurality of constants between the current time and future time.

[0038] According to the driver command predictor of technical solution 1, the controller includes:

[0039] A driver command interpreter configured to generate a plurality of intermediate commands in response to the plurality of actual driver commands and the plurality of future driver commands; and

[0040] A model predictive controller that responds to the plurality of intermediate commands and the plurality of future driver commands, which command the adjustment of the vehicle's one or more motion vectors at the plurality of update times.

[0041] According to the driver command predictor of technical solution 1, the future time is in the range of 100 milliseconds to 500 milliseconds after the current time.

[0042] A method for predicting driver commands includes:

[0043] In response to multiple actual driver commands and multiple future driver commands, the commands adjust one or more motion vectors of the vehicle relative to the road, wherein the multiple actual driver commands are received by the circuit at the current time;

[0044] In response to one or more motion vectors as commanded, multiple sensor data are generated to determine multiple actual states of the vehicle; and

[0045] In response to the driver model generating the plurality of future driver commands at multiple update times, wherein the plurality of update times range from the current time to a future time, and the driver model operates on the plurality of actual driver commands and the plurality of actual states to predict the plurality of future driver commands at the plurality of update times.

[0046] The method according to technical solution 10 further includes:

[0047] Generate road information, wherein the driver model operates on the road information to generate the plurality of future driver commands at the plurality of update times.

[0048] According to the method described in technical solution 10, the driver model is a deep learning model.

[0049] According to the method described in technical solution 12, the deep learning model is configured to generate multiple desired vehicle states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

[0050] According to the method of technical solution 13, the plurality of future driver commands are generated to make the plurality of desired states conform to the plurality of actual states.

[0051] The method according to technical solution 10 further includes:

[0052] Multiple results are generated by comparing the multiple future driver commands with the multiple actual driver commands; and

[0053] The driver model is updated based on the multiple results.

[0054] According to the method of technical solution 10, the plurality of actual driver commands are regarded as a plurality of constants between the current time and the future time.

[0055] A vehicle comprising:

[0056] Multiple driver controllers, configured to generate multiple actual driver commands; and

[0057] The driver command predictor is configured to:

[0058] In response to the plurality of actual driver commands and the plurality of future driver commands, the command adjusts one or more motion vectors of the vehicle relative to the road, wherein the plurality of actual driver commands are received at the current time;

[0059] In response to the one or more motion vectors as commanded, multiple sensor data are generated to determine multiple actual states of the vehicle; and

[0060] In response to the driver model generating the plurality of future driver commands at multiple update times, wherein the plurality of update times range from the current time to a future time, and the driver model operates on the plurality of actual driver commands and the plurality of actual states to predict the plurality of future driver commands at the plurality of update times.

[0061] According to the vehicle of technical solution 17, the driver command predictor further includes:

[0062] One or more information devices are configured to generate road information, wherein the driver model operates on the road information to generate the plurality of future driver commands at the plurality of update times.

[0063] According to the vehicle described in technical solution 18, wherein:

[0064] The driver model is a deep learning model; and

[0065] The deep learning model is configured to generate multiple desired vehicle states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

[0066] According to the vehicle of technical solution 19, the driver command predictor generates the plurality of future driver commands to make the plurality of expected states conform to the plurality of actual states.

[0067] The foregoing features and advantages, as well as other features and advantages, of this disclosure will become apparent when taken in conjunction with the accompanying drawings from the following detailed description of the best mode for carrying out this disclosure. Attached Figure Description

[0068] Figure 1 This is a schematic diagram illustrating a vehicle background according to one or more exemplary embodiments.

[0069] Figure 2 This is a schematic diagram of a driver command predictor according to one or more exemplary embodiments.

[0070] Figure 3 This is a schematic diagram of an example driver command according to one or more exemplary embodiments.

[0071] Figure 4 It is a graph of example driver commands within a limited range according to one or more exemplary embodiments.

[0072] Figure 5 This is a flowchart of a method for predicting driver commands according to one or more exemplary embodiments.

[0073] Figure 6 This is a flowchart of a driver model training method according to one or more exemplary embodiments.

[0074] Figure 7 This is a flowchart illustrating a learning technique for a driver model according to one or more exemplary embodiments.

[0075] Figure 8 This is a flowchart of a driving command prediction method according to one or more exemplary embodiments.

[0076] Figure 9 It is a flowchart of a method for calculating a desired state according to one or more exemplary embodiments.

[0077] Figure 10 This is a flowchart of a driver model adaptation method according to one or more exemplary embodiments.

[0078] Figure 11 This is a flowchart of a method for predicting driver commands according to one or more exemplary embodiments.

[0079] Figure 12 It is a graph of short-term steering wheel commands according to one or more exemplary embodiments. Detailed Implementation

[0080] Embodiments of this disclosure generally provide structures and / or techniques for predicting short-term driver commands to enhance vehicle predictive control. The structures and techniques predict short-term driver commands at each of multiple sampling times within a prediction range. Predictions may be based on current vehicle state, previous vehicle state, driver commands, road information, and / or sensor information available on the vehicle. Sensor information typically includes visual information, map information, radar information, etc. The predicted driver commands are then used to calculate enhancements to vehicle motion predictive control. Vehicle motion predictive control is redesigned to incorporate the predicted driver commands to achieve desired vector motions, particularly lateral and yaw movements, to support maximum lateral grip of the vehicle.

[0081] refer to Figure 1 The illustration shows a schematic diagram illustrating the background of a vehicle 80 according to one or more exemplary embodiments. The vehicle 80 moves on a road 82. The vehicle 80 includes a plurality of driver controllers 86, motion actuators 88, and a driver command predictor 100. The vehicle 80 can be occupied by a driver 84. The driver 84 is housed in the passenger compartment of the vehicle 80.

[0082] The actual driver command signal 102 can be generated by the driver 84 using the driver controller 86. The actual driver command signal 102 is received by the driver command predictor 100. The actual driver command signal 102 includes at least a steering component, an acceleration component, a braking component, and a gear selection component. The predicted motion command signal 104 is generated by the driver command predictor 100 and transmitted to the motion actuator 88. The predicted motion command signal 104 transmits at least a steering command, an acceleration command, a braking command, and a gear selection command to the motion actuator 88.

[0083] Vehicle 80 is implemented as an automobile (or passenger car). In various embodiments, vehicle 80 may include, but is not limited to, buses, trucks, autonomous vehicles, gas-powered vehicles, electric vehicles, hybrid vehicles, and / or motorcycles. Other types of vehicle 80 may be implemented to meet design standards for specific applications.

[0084] Vehicle 80 has vehicle dynamic characteristics 90. Vehicle dynamic characteristics 90 include acceleration characteristics 90a, braking characteristics 90b, and steering characteristics 90c. The motion of vehicle 80 relative to road 82 can be described as one or more motion vectors 92. Motion vectors 92 include longitudinal vector 92a, lateral vector 92b, and yaw vector 92c.

[0085] The driver 84 is the user of the vehicle 80. The driver 84 manually controls various functions in the vehicle 80. In various embodiments, the driver 84 can control steering, acceleration, braking, and gear selection by inputting manual commands into the driver controller 86.

[0086] The driver controller 86 implements multiple devices installed inside the vehicle 80 and used by the driver 84. The driver controller 86 is operable to provide input sensors and output indicators to the driver 84. The driver controller 86 may include a steering wheel, accelerator pedal, brake pedal, gear selector, speedometer, gear selection indicator, compass heading, etc. Other driver controllers 86 may be implemented to meet the design standards of specific applications.

[0087] The motion actuator 88 implements multiple electromechanical devices. The motion actuator 88 is operable to cause changes in the motion and orientation (or direction) of the vehicle 80 in response to a predicted motion command signal 104.

[0088] The driver command predictor 100 implements an adaptive predictive controller. The driver command predictor 100 is operable to command changes in the motion of the vehicle 80 within a finite prediction / control time range. This finite range can be from approximately tens of milliseconds (ms) (e.g., 100 ms) to hundreds of milliseconds (e.g., 500 ms). In various embodiments, the finite range can be 250 ms. Other durations can be implemented to meet the design criteria of a particular application.

[0089] In response to an actual driver command signal 102 and multiple future driver commands, the driver command predictor 100 commands a motion actuator 88 to adjust one or more motion vectors 92 of the vehicle 80 relative to the road 82. The actual driver command is received in the actual driver command signal 102 at the current time. In response to a driver model, future driver commands are generated internally within the driver command predictor 100 at multiple update times. The driver command predictor 100 also generates sensor data determining multiple actual states of the vehicle 80 in response to the commanded one or more motion vectors 92. The update times are between the current time and future times. The driver model operates on the actual driver commands and actual states of the vehicle 80 to predict future driver commands at the update times.

[0090] refer to Figure 2 The diagram illustrates an example embodiment of a driver command predictor 100 according to one or more exemplary embodiments. The driver command predictor 100 typically includes a controller 110, a plurality of sensors 112, a command prediction unit 114, a memory 116, and one or more information devices 118. The controller 110 includes a driver command interpreter 120 and a model prediction controller 122. The command prediction unit 114 includes a driver model 124. The memory 116 is operable to store a plurality of vehicle states 126.

[0091] The motion vector 92 of vehicle 80 is sensed by sensor 112. Command prediction unit 114 and driver command interpreter 120 receive actual driver command signal 102 from driver controller 86. Predicted motion command signal 104 is generated by model prediction controller 122 and presented to motion actuator 88.

[0092] Vehicle status signal 130 is generated by memory 116 and presented to command prediction unit 114 and model prediction controller 122. Vehicle status signal 130 carries vehicle status 126 (current state and past state). Road information signal 132 is generated by information device 118 and received by command prediction unit 114. Road information signal 132 transmits information collected by information device 118 about the road 82 and the surrounding environment of vehicle 80. This environment may include other vehicles, obstacles, weather, pedestrians, etc.

[0093] The future driver command signal 134 is generated by the command prediction unit 114 and presented to the driver command interpreter 120 and model prediction controller 122 in the controller 110. The future driver command signal 134 transmits the predicted future driver command to the controller 110. The intermediate command signal 136 is generated by the driver command interpreter 120 and presented to the model prediction controller 122. The intermediate command signal 136 carries a sampled driver command that remains constant within a finite range. The sensor signal 138 is generated by the sensor 112 and received by the memory 116. The sensor signal 138 carries sensor data generated by the sensor 112.

[0094] Controller 110 implements predictive circuitry. Controller 110 is operable to predict driver commands slightly into the future (e.g., 100 milliseconds to 500 milliseconds) and uses the prediction to induce changes in motion actuator 88. The changes requested from motion actuator 88 induce changes in the motion vector 92 of vehicle 80, taking into account vehicle dynamic characteristics 90.

[0095] Sensor 112 implements multiple electromechanical sensors. Sensor 112 is operable to convert changes in the physical motion of vehicle 80 into sensor data within sensor signal 138. The sensor data is processed and stored in memory 116 as vehicle state 126. Sensor 112 may include, but is not limited to, acceleration sensors and internal motion sensors.

[0096] Command prediction unit 114 implements short-term prediction technology. Command prediction unit 114 uses vehicle state 126, actual driver commands, and road information (past and present) to predict future driver commands within a limited range. The predicted driver commands are presented to controller 110 in future driver command signal 134.

[0097] Memory 116 implements a data storage device. Memory 116 is operable to store vehicle state 126 derived from sensor data and to present vehicle state 126 in vehicle state signal 130. In various embodiments, memory 116 may include non-volatile memory and / or volatile memory.

[0098] The driver command interpreter 120 determines the lateral and yaw motions of the vehicle 80 within the prediction / control range based on the vehicle state 126, the current driver command, and the future driver command. By considering the predicted future driver commands, the driver command interpreter 120 calculates more precise lateral and yaw motion control for the prediction / control range. The driver command interpreter 120 may assume that the current driver command remains constant (e.g., unchanged) within the prediction / control range.

[0099] The model predictive controller 122 is operable to generate a predicted motion command signal 104 based on a future driver command signal 134, an intermediate command signal 136, and a vehicle state 126. The combination of the future predicted driver commands enables the model predictive controller 122 to accurately control the motion of the vehicle 80 because it takes into account changes in driver commands.

[0100] Driver model 124 implements a deep learning model. Driver model 124 models the estimated behavior of driver 84. In various embodiments, driver model 124 is a neural network model. In other embodiments, driver model 124 is a recursive model.

[0101] refer to Figure 3 The diagram illustrates an example driver command according to one or more exemplary embodiments. The driver command may include a sequence of actual driver commands and predicted driver commands. The driver commands include at least a future steering command sequence 160, a braking command sequence 162, and an acceleration command sequence 164.

[0102] The actual driver command can be generated by the driver controller 86 and transmitted to the controller 110 and the command prediction unit 114 as an actual driver command signal 102. The actual driver command may include at least the current steering command δ received at the current time K. K Current braking command β K and the current acceleration command α K Other current driver commands can be implemented to meet the design criteria of specific applications.

[0103] The predicted driver commands can be generated by the command prediction unit 114 and presented to the controller 110 as a future driver command signal 134. The predicted driver commands may include a sequence of future steering commands δ at least at update times K, K+1, ..., K+P. K+1 to δK+P Future acceleration command sequence α K+1 To α K+P and the future braking command sequence β K+1 To β K+P The finite range extends from the current time K to a future time K+P. This is achieved by utilizing the current command δ. K β K and α K Both, controller 110 provides enhanced control performance. Additional predictive driver commands can be implemented to meet application-specific design criteria.

[0104] refer to Figure 4 The diagram illustrates a limited range of graphs 170 according to one or more exemplary embodiments. Graph 170 includes a first axis 172 and a second axis 174. The first axis 172 is plotted in time units spanning a prediction period 176 from the current time K to a future time K+P. The second axis 174 plots the amplitudes of various signals.

[0105] Curve 180 is a constant curve that illustrates the situation where the driver command predictor 100 determines what the driver 84 wants to happen in the absence of command prediction unit 114. Curve 180 illustrates the assumed constant command input from the driver 84.

[0106] Curve 182 is a reference curve illustrating the situation that the driver command predictor 100 determines the driver 84 wants to occur when considering the command prediction unit 114. Curve 184 illustrates the predicted output of the driver command predictor 100. Curve 186 illustrates the actual response of the vehicle 80 based on the predicted output in curve 184.

[0107] Curve 190 is an example of an actual driver command sequence. Curve 192 shows the current driver command treated as a constant. Curve 194 illustrates the predicted driver command generated by command prediction unit 114. Curve 196 illustrates the predictive control input generated by command prediction unit 114 and utilized by controller 110.

[0108] When the actual driver command is treated as a constant (constant curve 180), a first error 200 may exist between curve 180 and the actual output curve 186. When a command prediction unit 114 is included to estimate driver commands at several future time points, a second error 202 may exist between the predicted output curve 184 and the actual output curve 186. In this example, predicting and utilizing future driver commands results in a second error 202 that is smaller than the first error 200.

[0109] refer to Figure 5The following flowchart illustrates an example implementation of a method 210 for driver command prediction, according to one or more exemplary embodiments. Method (or process) 210 is implemented by a driver command predictor 100 and a non-vehicle computer. As shown, method 210 includes steps 212 through 224, utilizing a real-time feedback path 226 and an update feedback path 228. The sequence of steps is shown as a representative example. Other step sequences may be implemented to meet the criteria of a particular application.

[0110] Vehicle 80 can be considered a system that responds to inherent dynamics and driver commands. The motion actuator 88 of the driver assistance system 84 enhances vehicle handling performance. Vehicle motion control is limited by the capabilities of the motion actuator 88 and the tires.

[0111] The model predictive controller 122 provides real-time, optimal, and constrained solutions at multiple operating points. The driver model 124 helps anticipate driver steering and pedal commands and allows time-varying input estimates to be modeled within the predictive / control scope. Driver predictive inputs can also be used to recalculate state variables using the control vehicle dynamics equations when the driver command predictor 100 implements feedback control techniques. Furthermore, accurate handling predictions are provided that are further consistent with the calculated driver intent.

[0112] In method 210, a setup activity may be performed in step 212 to collect data for training driver model 124. The setup activity may be performed outside the vehicle 80 and offline. Driver model 124 is trained in step 214. This training typically involves a deep learning model tuned based on the behavior of driver 84 using an actual driver controller.

[0113] While the vehicle 80 is in motion, in step 216, sensor data is measured by sensor 112 to determine the vehicle state 126. In step 218, the driver command predictor 100 performs a short-term driver command prediction. The prediction is determined in real time within the prediction range. In step 220, the desired state calculation is performed based on the predicted driver command.

[0114] Predictive motion control calculations are performed in step 222. The goal of the calculations is to minimize the error between the actual state of vehicle 80 and the desired state of vehicle 80. The resulting error can be fed back to sensor measurement step 216 along real-time feedback path 226. Adaptation and refinement of driver model 124 can be performed in step 224. The refined driver model 124 can be used in step 218 to improve driver command predictions for each updated feedback path 228.

[0115] refer to Figure 6The diagram illustrates a flowchart of an example implementation of driver model training step 214 according to one or more exemplary embodiments. As shown, step 214 includes steps 242 to 250. The sequence of steps is shown as a representative example. Other step sequences may be implemented to meet the standards of a particular application.

[0116] In step 242, data collection may be performed to gather data related to the behavior of driver 84. In step 244, data may be collected within an appropriate timeframe to generate a large dataset. The data may include, but is not limited to, vehicle state 126, actual driver command signals 102, and road information signals 132. In step 246, the driver model 124 is trained and fine-tuned. Training includes preprocessing in step 248 and fine-tuning in step 250.

[0117] The data used for preprocessing step 248 can be provided via dataset 252 collected in step 244. Preprocessing step 248 includes feature generation, missing data handling, and outlier handling. Preprocessing step 248 generates processed data 254 for fine-tuning driver model 124. In step 250, driver model 124 is updated using the processed data 254. One or more iterations 256 of fine-tuning can be performed. The resulting fine-tuned driver model 124 is ready for use in vehicle 80 after the final iteration 256 has been completed.

[0118] refer to Figure 7 The diagram illustrates a flowchart of an example implementation of a learning technique 270 for a driver model 124 according to one or more exemplary embodiments. In this example, the learning technique 270 typically receives data from an information device 118 (e.g., a camera 118a, a lidar 118b, and other devices 118c). Other devices may include a mapping device, an inertial measurement unit, an electronic power steering torque sensor, wheel sensors, a steering wheel sensor, a suspension height sensor, a global positioning system, a throttle sensor, a brake pedal sensor, radar, an accelerometer, and / or other estimation signals obtained from various sensors.

[0119] Learning technique 270 can perform one or more convolutions 272 on spatial data received from camera 118a and lidar 118b. Fine-tuning step 250 can be used to convolve the data and data received from other devices 118c to build a cyclic and fully connected driver model 124.

[0120] refer to Figure 8 The flowchart illustrates an example implementation of driving command prediction step 218 according to one or more exemplary embodiments. Step 218 includes steps 280 and 282.

[0121] In step 280, data is measured from the driver controller 86, sensor 112, and information device 118. In this example, the data includes actual driver commands 102, road information 132, and sensor data. The sensor data determines the vehicle state 126.

[0122] Preprocessing step 248 is performed to determine features, handle missing sensor data points, and process anomalous data. The processed data 254 is then presented to command prediction unit 114. In step 282, the driver model 124 in command prediction unit 114 generates predicted driver command results 284. The predicted driver command results 284 include future steering commands (e.g., curve 290), future throttle commands (e.g., curve 292), future braking commands (e.g., curve 294), future gear commands (e.g., curve 296), etc. The predicted driver command results 284 can span prediction periods 176 in real time (e.g., K to K+P).

[0123] refer to Figure 9 The flowchart illustrates an example implementation of the desired state calculation step 220 according to one or more exemplary embodiments. As shown, step 220 includes steps 300 to 314. The sequence of steps is shown as a representative example. Other step sequences may be implemented to meet the standards of a particular application.

[0124] In step 300, predicted positions of the desired accelerator pedal and brake pedal are calculated. In step 302, estimates of the drive torque, braking torque, and wheel speed at each corner of the vehicle 80 (e.g., at each tire) are calculated. In step 304, the slip ratio is predicted. In step 306, the combined longitudinal slip force is predicted.

[0125] In step 308, a steering prediction is calculated. In step 310, a slip angle (e.g., the vehicle's state) is predicted. In step 312, combined slip lateral forces on the tires are calculated. In various embodiments, steps 308 to 312 may be performed in parallel with steps 300 to 306. In other embodiments, steps 300 to 312 may be performed sequentially. In step 314, the desired lateral and longitudinal vehicle movements may be predicted based on the vehicle dynamics control equations. To provide more precise control actions, the effects of changes in driver commands may be considered within the prediction range. If the driver command changes, the desired vehicle state is recalculated based at least on the predicted steering, throttle, and braking inputs.

[0126] The desired state is calculated based on the driver's predicted command. The predicted wheel speed (Iw) can be determined according to Equation 1 as follows:

[0127] Equation (1)

[0128] Where ω is the wheel speed, T is the torque at the corner (wheel), and R... eff It is the effective radius of the tire, and F x It is the longitudinal force of the wheel.

[0129] The corner slip ratio (ĸ) is defined by Equation 2 as follows:

[0130] Equation (2)

[0131] Among them, υ x It is the forward velocity, and R e It is the effective radius of the tire.

[0132] The lateral slip angle (α) can be defined by Equation 3 as follows:

[0133] Equation (3)

[0134] Where υ y It is the lateral velocity.

[0135] The tire / road contact surface plane force is calculated according to equations 4 and 5, as shown below:

[0136] Equation (4)

[0137] Equation (5).

[0138] The vehicle body dynamics control equations 6-8 are provided as follows:

[0139] Equation (6)

[0140] Equation (7)

[0141] Equation (8)

[0142] Among them, G z It is the torque about the z-axis, L f It is the distance from the front axle to the center of gravity, L r It is the distance from the rear axle to the center of gravity, L w δ is the wheel gauge, and δ is the wheel angle of the front axle.

[0143] The desired longitudinal and lateral states of the vehicle are defined by Equation 9-11 as follows:

[0144] Equation (9)

[0145] Equation (10)

[0146] Equation (11)

[0147] Where r is the yaw rate, and I z It is the moment of inertia.

[0148] For predictive motion control, a target can be set to minimize the actual state. and expected state The error between them. Therefore, the following quadratic cost function can be defined within a finite time frame. J MPC So that the desired vehicle motion can be achieved according to Equation 12 as follows:

[0149] Equation (12)

[0150] According to equations 13-16:

[0151] Equation (13)

[0152] Equation (14)

[0153] Equation (15)

[0154] Equation (16)

[0155] Where J is the cost function, U is the control action (or control effort), y represents the output of the vehicle model, x represents the state of the vehicle model, and A, B, and W are matrices representing the state space of the vehicle model.

[0156] In the driver command predictor 100, the expected vehicle motion is updated within the prediction range.

[0157] refer to Figure 10 The flowchart illustrates an example implementation of driver model adaptation step 224 according to one or more exemplary embodiments. Step 224 includes steps 320 to 324, which are linked by predicted driver command 326, state 328, and reward 330.

[0158] In step 224, the predicted driver command signal 104 and the actual driver command signal 102 are compared. If the driver 84's response differs from the prediction (e.g., exceeding a corresponding threshold), a reinforcement learning (RL) mechanism can be initialized in real time to adapt and improve the driver model 124. The reinforcement learning mechanism learns the behavior of the driver 84 and customizes the driver model 124 for each vehicle 80 / driver 84 for unseen behaviors. Abnormal / bad driving behaviors can be discarded.

[0159] In step 320, the driver model 124, acting as a proxy, generates a predicted driver command 326. In step 322, the predicted driver command 326 is compared with the actual driver command to determine state 328. In step 324, a prediction error is generated as the difference between the predicted driver command 326 and the actual driver command. The prediction error is returned to step 320 in reward 330. Step 320 then adjusts the driver model 124 based on state 328 and reward 330.

[0160] refer to Figure 11 The following flowchart illustrates an example implementation of a method 340 for predicting driver commands, according to one or more exemplary embodiments. Method (or process) 340 is implemented in a driver command predictor 100. As shown, method 340 includes steps 342 to 354. The sequence of steps is shown as a representative example. Other step sequences may be implemented to meet the criteria of a particular application.

[0161] Step 342 includes, in response to the actual driver command signal 102 and the future driver command signal 134, commanding the adjustment of one or more motion vectors 92 of the vehicle 80 relative to the road 82. The controller 110 receives the actual driver command at the current time K. The controller 110 receives the future driver command at update times K to K+P. The update time range is from the current time K to the future time K+P.

[0162] Step 344 generates sensor data to determine vehicle state 126 in response to the commanded motion vector 92. Road information is generated in step 346. In step 348, driver model 124 operates on the actual driver command, vehicle state 126, and road information to generate future driver commands at each update time K to K+P.

[0163] Step 350 includes generating the desired state of vehicle 80 based on actual driver commands and vehicle dynamic characteristics 90. Generate commands for the future driver to achieve the desired state. Conforms to actual conditions In step 352, a result is generated by comparing the future driver command with the actual driver command. In step 354, the driver model 124 is updated based on the result generated in step 352. Method 340 is then repeated to maintain the intended motion of the vehicle 80.

[0164] refer to Figure 12 A graph 360 predicting short-term steering wheel commands is shown according to one or more exemplary embodiments. Graph 360 includes a first axis 362 and a second axis 364. The first axis 362 illustrates time in seconds. The second axis 364 illustrates steering wheel angles in degrees.

[0165] Curve 360 ​​illustrates a driving scenario where vehicle 80 travels at a variable speed along a curved road 82. Curve 366 shows the actual steering wheel angle command controlled by driver 84. Curve 368 illustrates the predicted steering wheel angle command calculated by driver command predictor 100. The results of the driving scenario show that the predicted steering wheel angle (curve 368) approximately matches the actual value (curve 366) within a range of less than 2 degrees.

[0166] In extreme handling scenarios (e.g., highly dynamic maneuvering, slippery surfaces, etc.), the driver command predictor 100 supports vehicle 80 in achieving maximum lateral grip by utilizing most of the tire capacity in the front and rear axles. To this end, the desired lateral acceleration can be calculated at each sampling time using driver steering commands and brake / throttle commands. The driver command predictor 100 predicts short-term driver commands within the prediction range and during each sampling time, based on the current and previous vehicle states, actual driver commands, road information, and available sensor information about the target vehicle 80. Using the predicted driver commands, the driver command predictor 100 calculates control adjustments to achieve the acceleration requested by driver 84, and particularly the lateral acceleration, in a way that vehicle 80 achieves maximum lateral grip while maintaining vehicle stability and reducing frequent reverse steering in severe extreme handling driving scenarios.

[0167] In various embodiments, the driver command predictor 100 provides predictions of short-term driver commands to enhance vehicle predictive motion control. The driver command predictor 100 includes a controller 110 that commands adjustments to a motion vector 92 of the vehicle 80 relative to a road 82 in response to actual driver commands and future driver commands. Driver commands are received at the current time. Future driver commands are received at multiple update times ranging from the current time to future times. These future driver commands are used to compute enhancements to the automated vehicle motion predictive control, providing more realistic navigation control than assumed constant driver decisions. Sensors 112 generate sensor data determining multiple actual states of the vehicle 80 in response to the motion vector 92. A prediction unit 114 is configured to generate future driver commands at update times in response to a driver model 124. The driver model 124 operates on actual driver commands and actual states to predict future driver commands at update times. Vehicle motion predictive control is automatically re-formulated to incorporate the predicted driver commands to achieve desired lateral and yaw movements to support maximum lateral grip of the vehicle.

[0168] All parameter values ​​(e.g., quantities or conditions) in this specification, including the appended claims, should be understood to be modified in all cases by the term "about," regardless of whether "about" actually appears before the value. "About" indicates that the value allows for some slight imprecision (the value is close to exact; approximately or reasonably close to the value; nearly). If the imprecision provided by "about" is not to be understood in the ordinary sense in the art, then "about" as used herein at least indicates the variation that might arise from ordinary methods of measuring and using these parameters. Furthermore, the disclosure of ranges includes the disclosure of all values ​​throughout the range and further subdivisions of the range. Each value within the range and the endpoints of the range are disclosed as separate embodiments.

[0169] While the best mode of implementing this disclosure has been described in detail, those skilled in the art will recognize various alternative designs and embodiments for implementing this disclosure within the scope of the appended claims.

Claims

1. A driver command predictor, comprising: A controller is configured to command the adjustment of one or more motion vectors of the vehicle relative to a road in response to a plurality of actual driver commands and a plurality of future driver commands, wherein the plurality of actual driver commands are received at the current time, the plurality of future driver commands are received at a plurality of update times, and the plurality of update times range from the current time to future times. Multiple sensors, configured to generate multiple sensor data to determine multiple actual states of the vehicle in response to one or more motion vectors as commanded; and A command prediction unit is configured to generate a plurality of future driver commands in response to a driver model generating the plurality of future driver commands at multiple update times, wherein the driver model operates on the plurality of actual driver commands and the plurality of actual states to predict the plurality of future driver commands at the plurality of update times. The controller includes: A driver command interpreter configured to generate a plurality of intermediate commands in response to the plurality of actual driver commands and the plurality of future driver commands; and A model predictive controller that responds to the plurality of intermediate commands and the plurality of future driver commands, which command the adjustment of the vehicle's one or more motion vectors at the plurality of update times.

2. The driver command predictor according to claim 1, further comprising: One or more information devices are configured to generate road information, wherein the driver model operates on the road information to generate the plurality of future driver commands at the plurality of update times.

3. The driver command predictor according to claim 1, wherein, The driver model is a deep learning model.

4. The driver command predictor according to claim 3, wherein, The deep learning model is configured to generate multiple desired states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

5. The driver command predictor according to claim 4, wherein, The command prediction unit generates multiple future driver commands to make the multiple expected states conform to the multiple actual states.

6. The driver command predictor according to claim 1, wherein, The command prediction unit is also configured to: Multiple results are generated by comparing the multiple future driver commands with the multiple actual driver commands; as well as The driver model is updated based on the multiple results.

7. The driver command predictor according to claim 1, wherein, The controller treats the multiple actual driver commands as multiple constants between the current time and future time.

8. The driver command predictor according to claim 1, wherein, The future time is in the range of 100 to 500 milliseconds after the current time.

9. A method for predicting driver commands, comprising: In response to multiple actual driver commands and multiple future driver commands, the command adjusts one or more motion vectors of the vehicle relative to the road, including: generating multiple intermediate commands in response to the multiple actual driver commands and the multiple future driver commands; and adjusting the one or more motion vectors of the vehicle at multiple update times in response to the multiple intermediate commands and the multiple future driver commands, wherein the multiple actual driver commands are received by circuitry at the current time; In response to one or more motion vectors as commanded, multiple sensor data are generated to determine multiple actual states of the vehicle; and In response to the driver model generating the plurality of future driver commands at multiple update times, wherein the plurality of update times range from the current time to a future time, and the driver model operates on the plurality of actual driver commands and the plurality of actual states to predict the plurality of future driver commands at the plurality of update times.

10. The method of claim 9, further comprising: Generate road information, wherein the driver model operates on the road information to generate the plurality of future driver commands at the plurality of update times.

11. The method according to claim 9, wherein, The driver model is a deep learning model.

12. The method according to claim 11, wherein, The deep learning model is configured to generate multiple desired states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

13. The method according to claim 12, wherein, Generate the plurality of future driver commands to make the plurality of desired states conform to the plurality of actual states.

14. The method of claim 9, further comprising: Multiple results are generated by comparing the multiple future driver commands with the multiple actual driver commands; and The driver model is updated based on the multiple results.

15. The method according to claim 9, wherein, The multiple actual driver commands are considered as multiple constants between the current time and the future time.

16. A vehicle comprising: Multiple driver controllers are configured to generate multiple actual driver commands; and The driver command predictor is configured to: In response to the plurality of actual driver commands and the plurality of future driver commands, the command adjusts one or more motion vectors of the vehicle relative to the road, including: generating a plurality of intermediate commands in response to the plurality of actual driver commands and the plurality of future driver commands; and adjusting the one or more motion vectors of the vehicle at the plurality of update times in response to the plurality of intermediate commands and the plurality of future driver commands, wherein the plurality of actual driver commands are received at the current time; In response to the one or more motion vectors as commanded, multiple sensor data are generated to determine multiple actual states of the vehicle; and In response to the driver model generating the plurality of future driver commands at multiple update times, wherein the plurality of update times range from the current time to a future time, and the driver model operates on the plurality of actual driver commands and the plurality of actual states to predict the plurality of future driver commands at the plurality of update times.

17. The vehicle according to claim 16, wherein, The driver command predictor also includes: One or more information devices are configured to generate road information, wherein the driver model operates on the road information to generate the plurality of future driver commands at the plurality of update times.

18. The vehicle according to claim 17, wherein: The driver model is a deep learning model; and The deep learning model is configured to generate multiple desired states based on the multiple actual driver commands and multiple vehicle dynamic characteristics.

19. The vehicle according to claim 18, wherein, The driver command predictor generates the plurality of future driver commands to make the plurality of expected states conform to the plurality of actual states.

Images