Humane Steering Model for Automated Vehicles

Abstract

A humanized steering system (10) for an automated vehicle (12) includes one or more steering wheels (16) operable to steer the vehicle (12), an angle sensor (20) configured to determine a steering angle (18) of the steering wheel (16), a hand wheel (22) used by an operator (14) of the vehicle (12) to affect the steering angle (18) and thereby manually steer the vehicle (12), a steering actuator (24) operable to affect the steering angle (18) and thereby steer the vehicle (12) when the operator (14) is not manually steering the vehicle (12), a position sensor (26) operable to indicate a relative position (28) of an object (30) proximate the vehicle (12), and a controller (32). The controller (32) is configured to receive the steering angle (18) and the relative position (28), determine a steering model (34) based on the steering angle (18) and the relative position (28) using deep learning techniques, and operate the steering actuator (24) to steer the vehicle (12) according to the steering model (34) when the operator (14) is not manually steering the vehicle (12), thereby steering the vehicle (12) in a manner similar to how the operator (14) manually steers the vehicle (12).

Description

[0001] This application is a divisional application of the invention patent application with an international application date of 2016-08-24, international application number of PCT / US2016 / 048331, application number 201680058254.6 entering the Chinese national phase, and titled “Humanized Steering Model for Automated Vehicles”. Technical Field

[0002] The present disclosure relates generally to humanized steering systems for automated vehicles, and more particularly to using deep learning techniques to develop a steering model that steers a vehicle in a manner similar to how an operator manually steers the vehicle. Background Art

[0003] Fully automated (i.e., autonomous) vehicles that steer the vehicle without any manual input from the operator of the vehicle have been proposed. However, the algorithms that control the steering of the vehicle have performance tuning variables that complicate the algorithms for tuning passenger comfort. Even when passenger comfort is taken into account, the performance of the algorithms only provides excellent comfort during the specific steering scenarios for which the algorithms are tuned. What is needed is an algorithm or steering model that mimics the steering characteristics of a human operating the vehicle. Summary of the invention

[0004] According to one embodiment, a humanized steering system for an automated vehicle is provided. The system includes one or more steering wheels operable to steer the vehicle. The system also includes an angle sensor configured to determine the steering angle of the steering wheel. The system also includes a hand-wheel used by an operator of the vehicle to affect the steering angle and thereby manually steer the vehicle. The system also includes a steering actuator operable to affect the steering angle, thereby steering the vehicle when the operator does not manually steer the vehicle. The system also includes a position sensor operable to indicate the relative position of an object approaching the vehicle. The system also includes a controller configured to receive the steering angle and the relative position. The controller is further configured to use deep learning technology to determine a steering model based on the steering angle and the relative position, and to operate the steering actuator to steer the vehicle according to the steering model when the operator does not manually steer the vehicle, thereby steering the vehicle in a manner similar to how the operator manually steers the vehicle.

[0005] Further characteristics and advantages will become more apparent on reading the following detailed description of a preferred embodiment, given only as a non-limiting example, and with reference to the various accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0007] Figure 1 is a diagram of a humanized steering system for an automated vehicle according to one embodiment;

[0008] Figure 2 is a data table in a system according to one embodiment;

[0009] Figure 3 According to an embodiment Figure 1 a graph of the data in the system; and

[0010] Figure 4 According to one embodiment, Figure 1 A logical diagram of the data of the processes performed by the system. DETAILED DESCRIPTION

[0011] Figure 1 A non-limiting example of a humanized steering system, hereinafter referred to as system 10, is shown. Although the description of system 10 presented herein is generally directed to a fully automated (i.e., autonomous) vehicle, hereinafter referred to as vehicle 12, it is contemplated that the teachings presented herein are applicable to partially automated vehicles that can momentarily take over control of vehicle 12 or momentarily assist an operator 14 in manually steering vehicle 12 to, for example, avoid another vehicle.

[0012] The system 10, or more specifically the vehicle 12, is equipped with one or more steerable wheels, hereinafter referred to as steerable wheels 16. The steerable wheels 16 may generally be characterized as being operable to steer the vehicle 12. While it is contemplated that the vehicle 12 is typically a four-wheeled automobile, it is contemplated that the teachings presented herein will also be applicable to two-wheeled or three-wheeled vehicles equipped with a single steerable wheel, articulated vehicles that steer by changing the relative angle between the front and rear of the vehicle, and other wheeled vehicles that may be equipped with more than four wheels and / or more than two steerable wheels.

[0013] In order for the system 10 to determine the angle or direction of the steering wheel 16, the system includes an angle sensor 20 configured to determine and / or indicate a steering angle 18 of the steering wheel 16. The angle sensor may output or provide a signal indicative of the steering angle 18 using one of a variety of technologies, such as a variable resistor, a variable reluctance, or a digital position encoder.

[0014] As will be explained in greater detail later, the system 10 described herein is generally configured to operate or steer a vehicle 12 in a manner that mimics a human (i.e., operator 14) manually steering the vehicle 12. Typically, the system "learns" the manner in which the operator 14 manually steers the vehicle 12 by observing / recording various aspects that affect how the operator 14 may steer the vehicle 12, and then building a mathematical model in an effort to replicate the observed behavior.

[0015] In order to enable the operator 14 to manually steer the vehicle 12, the system 10 (or vehicle 12) is equipped with a hand wheel 22 that is used or operable by the operator 14 of the vehicle 12 to affect the steering angle 18, thereby manually steering the vehicle 12. Although the term hand wheel is generally interpreted as a relatively round hand wheel typically found in automobiles, other alternative steering means such as joysticks, handles, reins, foot pedals, buttons, knobs or sliding knobs, etc. are also contemplated and included in the definition of hand wheel 22.

[0016] The system 10 also includes a steering actuator 24 that can be used or operable to affect the steering angle 18 so that the system 10 is able to steer the vehicle 12 when the operator 14 is not manually steering the vehicle 12. In one embodiment, the steering actuator 24 can cause the hand wheel 22 to move as the steering angle 18 changes, which is typically the case for assisted steer-type steering mechanisms. Alternatively, the steering mechanism can be characterized as a steer-by-wire steering mechanism in which there is no mechanical connection between the hand wheel 22 and the steering actuator 24. That is, in a steer-by-wire system, there may only be an electrical connection, so when the system 10, rather than the operator 14, is steering the vehicle 12, the hand wheel 22 may not move in response to changes in the steering angle 18.

[0017] The system 10 also includes a position sensor 26 operable to indicate a relative position 28 of an object 30 proximate to the vehicle 12. Non-limiting examples of the object 30 include lane markings on the surface of a road or travel lane on which the vehicle 12 is traveling, road edges, signs, sign poles, barriers such as guardrails, construction barrels, pedestrians, animals, and other vehicles such as automobiles, motorcycles, bicycles, etc. The position sensor 26 may include, but is not limited to, one or any combination of a camera 26A, a radar unit 26B, and a lidar unit 26C.

[0018] The system 10 also includes a controller 32 that is configured to receive, for example, the steering angle 18, the relative position 28, and / or the manual control signal 38 for observing / learning how the operator 14 operates (e.g., steers) the vehicle 12 when the operator 14 is manually operating the vehicle 12. The controller 32 is also configured to operate or steer the vehicle 12 in a manner similar to how the operator 14 manually steers the vehicle 12 when the operator 14 is not manually operating or steering the vehicle 12.

[0019] The controller 32 may include a processor (not specifically shown) such as a microprocessor or other control circuits, such as analog and / or digital control circuits, including application specific integrated circuits (ASICs) for processing data as should be apparent to those skilled in the art. The controller 32 may include a memory (not specifically shown) to store one or more routines, thresholds, and captured data, including non-volatile memory, such as an electrically erasable programmable read-only memory (EEPROM). As described herein, the one or more routines may be executed by the processor to perform steps for determining the manner in which the operator 14 manually operates or steers the vehicle 12 and autonomously operates the vehicle 12 in a similar manner.

[0020] Given information present, for example, in the steering angle 18 from the angle sensor 20, the relative position 28 about the object 30 from the position sensor 26, and other information (e.g., size, shape, motion, classification, etc.), the controller is configured to determine a steering model 34 based on at least the steering angle 18 and the relative position 28 by using deep learning techniques. The use of deep learning techniques is advantageous over control theory-based techniques because deep learning relaxes some of the preconceived notions or assumptions necessary for control theory-based model building techniques. Deep learning techniques readily include the effects of real-time conditions, such as, for example, how temperature may affect tire stiffness and / or how steering maneuvers affect passenger comfort. The steering model 34 may be constructed or trained using a recursive neural network (RNN), which is a time-based deep learning method that can be implemented in a dynamic system.

[0021] The steering model 34 has an input data structure, which can be characterized as a binary vector 36, which includes multiple inputs such as weather, obstacles, vehicle status and / or vehicle position in a lane. Each bit or bit combination in the binary vector represents an element of the input. Once the steering model 34 is determined, when the operator 14 is not manually steering the vehicle 12, the system 10 is able to operate the steering actuator 24 to steer the vehicle 12 according to the steering model 34. This provides a means by which the vehicle 12 is steered in a manner similar to how the operator 14 manually steers the vehicle 12. As an example and not limitation, when the operator 14 is not manually steering the vehicle 12, the steering model 34 can output a desired angle 40 of the steering wheel 16 to steer the vehicle 12. As will be appreciated by those skilled in the art, the control of the steering actuator 24 in response to the changing value of the desired angle 40 can be through a proportional-integral-derivative (PID) position control algorithm. Alternatively, the response time characteristics of the steering actuator 24 can be included in the steering model 34. In alternative non-limiting embodiments, the desired angle 40 may instead be a desired torque value that is output by the controller 32 and used to determine, for example, how much current to apply to the steering actuator 24 so that the steering actuator 24 generates or outputs to change the steering angle 18 of the steered wheels 16 .

[0022] Figure 2 A non-limiting example of a binary vector 36 or a portion thereof is shown. As described above, the steering model 34 may be represented by a binary vector 36, which may be composed of a plurality of bits indicating the parameters 42 of the steering model 34. Each bit of the binary vector 36 may correspond to an element of the input. For example, the bits forming the binary vector 36 may indicate various weather conditions and use 1s and 0s to indicate those weather conditions. Figure 2 is a suitable example of how a pleasant sunny day may be indicated. For the indicated weather condition 42A, the steering model 34 model will assume that the object 30 is easily or more confidently detected / classified by the camera 26A and that traction is good. Conversely, when the weather condition 42A indicates that rain or snow is showing, the operator 14 may manually steer the vehicle 12 in a more cautious manner, and thus the steering model 34 may be adjusted to steer the vehicle 12 less aggressively when rain or snow conditions are indicated.

[0023] Despite Figure 2 4A may include temperature 44. Temperature 44 may be indicated in binary vector 36 by an eight-bit number (000000000 to 11111111), where zero (0) corresponds to less than or equal to -65°C and 11111111 corresponds to greater than or equal to 190°C. Steering model 34 may use temperature 44 to help determine, for example, whether ice may have formed on the road, or to estimate the stiffness of the tires of vehicle 12 and adjust parameters 42 of binary vector 36 such as steering response time 46 of vehicle specification 42C.

[0024] The binary vector 36 may include one or more parameters 42 indicative of a vehicle state 42B, such as, but not limited to, a lateral acceleration 48A and / or a longitudinal acceleration 48B ( Figure 1 lateral / longitudinal acceleration in the vehicle), vehicle speed 50, yaw rate 52, and steering angle 18. Thus, the system 10 may include or be equipped with an accelerometer 54 and / or a speed sensor 56. These variables may be indicated by a binary number input into the binary vector 36. For example, the vehicle speed 50 may be indicated by a seven-bit binary number, so speeds from zero (0000000) to greater than or equal to one hundred and twenty-seven (1111111) miles per hour may be indicated.

[0025] The binary vector 36 may include one or more parameters 42 indicative of vehicle specifications 42C, such as, but not limited to, vehicle dimensions 58A (e.g., wheelbase and / or body length), vehicle mass 58B, steering response time 58D, and / or engine power 58C. If the system 10 identifies an instance of an object 30 that requires steering around (i.e., avoiding), all of these variables may affect how the vehicle 12 responds to changes in the steering angle 18 and thereby affect what the operator 14 senses with respect to comfort and human-like steering behavior of the system 10.

[0026] The binary vector 36 may include one or more parameters 42 indicating object characteristics 42D, such as, but not limited to, object position 60A, object vector 60B, object classification 60C, and lane position 60D. As described above, object 30 may be any object present on or near the road. Object 30 may be something of a specific biased value used by system 10 to steer the vehicle to lane position 60D. That is, lane position 60D may not be the center of the selected driving lane. Alternatively, object 30 may be within the forward driving path of vehicle 12, or moving in a direction (object vector 60B) that will intersect the forward driving path.

[0027] As a further example, the object position 60A may be indicated in a binary vector by which of eight positions or orientations the object 30 occupies with respect to the vehicle 12. The eight bits may correspond to Figure 3 , which can be respectively labeled in the binary vector 36 as<Dir_LF> ,<Dir_F> ,<Dir_RF> ,<Dir_L> ,<Dir_R> ,<Dir_LB> ,<Dir_B> ,<Dir_RB> If the object 30 is larger, such as a semi / trailer that may occupy the right front, right and right rear directions or positions, the binary object position 60A may be 00101001.

[0028] Object classification 60C may be indicated by a combination of bits to indicate various information about the obstacle including classification, distance, and / or speed (if the object 30 is moving). As an example, 0000 indicates no obstacle, 0001 indicates a car, 0010 indicates a person, 0011 indicates an animal, and 0100 indicates a roadblock. Distance and speed may also be indicated in binary form in the binary vector 36. Similarly, with respect to lane position 60D, 000 indicates "in the middle", 101 indicates "left side of the road", 001 indicates "right side of the road", 110 indicates "double lanes on the left", and 011 indicates "double lanes on the right".

[0029] The steering model 34 can be implemented as a discrete time model with a sampling / update rate of, for example, ΔT=0.1 seconds. If the data collection frequency is different, the sampling / update rate can be modified. At time T[i], the binary vector 36 (i.e., the input for the steering model 34) can be expressed as X_T[i]. The steering model 34 includes a transformation function H_T[i], which is sometimes referred to as a hidden layer and is characterized as being composed of neurons. That is, H_T[i] here is the hidden layer at time T[i] with a new input X_T[i]. As a further explanation, the neurons in the hidden layer are like neurons in the human brain, "grabbing" data from the input and extracting features or parameters from the input so that the brain will learn how to recognize / classify information and react to situations. The hidden layer is essentially invisible, but very important because the learning process occurs here.

[0030] The output of the steering model 34 at time T[i] can be expressed as Y_T[i]. The input (binary vector 36), the hidden layer H_T[i] and the output Y_T[i] can be connected or related to each other through the weight matrices WI, WH and WO. The hidden layer H_T[i] may have "grabbed" some features that will affect the future value of the desired angle 40. Therefore, H_T[i] is taken into account when calculating the new hidden layer at time T_[i+1]. Given the input X_T[i+1] at time T[i+1], H_T[i+1]=f(WI*X_T[i+1]+WH*H_T[i]), where f is the activation function. Typically, f(x)=tanh, or f(x)=1 / (1+exp(-x)). Therefore, the output at T[i+1] is: Y_T[i+1]=fo(WO*(H_T[i+1])), where fo is the activation function and usually fo=tanh or fo is a sigmoid function.

[0031] RNN is characterized as a supervised learning algorithm, which means that the input data forming the binary vector 36 is labeled. Labeled data means that given an input, the desired output is given, thus driving the learning process towards an acceptable solution. In this example, there is a true output Yd_T[i] at time T[i]. However, for unsupervised learning, the data is unlabeled, that is, no response is provided. The system 10, or more specifically the controller 32, needs to classify the data itself. Once classified, the system 10 uses the "timed backpropagation (backpropagation through time)" method to adjust the weight matrices WI, WH and WO. The backpropagation method is used to minimize the error function by gradient descent.

[0032] At time T[i], the step error is expressed as E_T[i]=sum[(Yd_T[m]-Y_T[m])^2], m=1,2,...L, where L is the number of neurons in the output layer. At time T[n], the cumulative error is defined as the sum of the step errors at previous times, E_a=sum(E_T[i]), i=1,2,3,...n. Equations 1-3 illustrate the computation of the derivation using the chain rule to apply timed backpropagation:

[0033]

[0034] as well as

[0035]

[0036] Using gradient descent to update WI, WH, and WO provides Equations 4-6.

[0037] new

[0038] new as well as

[0039] new

[0040] Where η is the learning rate.

[0041] The problem with RNNs is the vanishing gradient problem mentioned by Yoshua Bengio, Patrice Simard and Paolo Frasconi in a publication entitled "Learning Long-Term Dependencies with Gradient Descent is Difficult" published by Bengio, Smiard and Frasconi in IEEE transactions on Neural Networks, Volume 5, No. 2, March 1994. If this problem occurs during training, long short-term memory (LSTM) can be used to avoid it. The LSTM method given below is adopted from the online lecture notes (Lecture 12) published by the Department of Computer Science at the University of Oxford, which is the course material for machine learning in the 2014-2015 academic year.

[0042] Figure 4 A non-limiting example of a logic diagram 400 is shown for illustrating a process performed by the controller 32. The operations in the logic diagram 400 correspond to equations 7-12.

[0043] i_T[i+1]=f(Wxi*x_T[i+1]+Whi*h_T[i]), Equation 7,

[0044] f_T[i+1]=f(Wxf*x_T[i+1]+Whf*h_T[i]), Equation 8,

[0045] o_T[i+1]=f(Wxo*x_T[i+1]+Who*h_T[i]), Equation 9,

[0046] g_T[i+1]=g(Wxg*x_T[i+1]+Whg*h_T[i]), Equation 10,

[0047] c_T[i+1]=i_T[i+1]●g_T[i+1]+f_T[i+1]●c_T[i] Equation 11,

[0048] h_T[i+1]=o_T[i+1]●h(c_T[i+1]) Equation 12,

[0049] Wherein functions f, g, h are activation functions, which may be sigmoid functions, tanh, and ● represents element-wise multiplication.

[0050] Repeat the above process for the new h_T[i+1]. Here Wxi, Whi, Wxf, Whf, Wxo, Who, Wxg and Whg are used instead of WI, WH. Use timed backpropagation to adjust the weight matrix. In practice, there are several ways to affect the learning performance. The number of neurons in the hidden layer is uncertain and it affects the training results. In order to achieve the best results, the data is divided into two parts: training set and test set. Use the training set to apply the above algorithm to obtain the weights and use the test set to obtain the errors. Adjust the number of neurons and repeat the learning process until the error reaches a minimum. Another way to improve the results can be to add biases and adjust the biases during backpropagation. Adding more hidden layers in this algorithm is another way to improve the performance. Adding more hidden layers means that more previous time steps will be considered for the next step.

[0051] The system 10 learns to steer the vehicle 12 in a manner similar to a human (e.g., operator 14). To avoid learning bad driving habits, there are two options: one is to apply a filter to ignore bad driving times and keep good driving times to learn; the other is to record data from good drivers. Or combine the two methods to achieve the best results.

[0052] The steering model described herein provides a system 10 that can steer a vehicle 12 in a manner that mimics an operator. Further understanding of the system 10 is provided by the description of several "use cases" described below. For each use case, the inputs may be different, but not all inputs need to be considered for each case.

[0053] The simplest case is lane keeping. In this case, the inputs can be defined as simple as: the distance to the lane boundary (lane position 60D), the current value of the steering angle 18, and the current value of the vehicle speed 50. Generally, it is known that the control algorithm tends to steer the vehicle toward the center of the lane of travel. However, different people have different driving habits. The output of the steering model 34 does not need to keep the vehicle in the center. It is more similar to the operator 14 whose data is used for training.

[0054] Lane changes are also common. In addition to inputs for lane keeping; inputs for obstacle classification, obstacle size, and distance to object 30 may also need to be considered. Note that different people handle driving situations differently; some drivers may want to avoid obstacles as early as possible, while some drivers tend to change lanes quickly when approaching obstacles. In general, there is no fixed standard to judge the driving habits of operators 14 as long as it is safe for themselves and others. By using different training data, the trained model can be better adjusted to meet the habits of different people.

[0055] Another scenario is similar to what exists on a highway: a constant non-zero curvature path, and also includes points with discontinuous curvature as the vehicle transitions from the highway to a slope. In this case, comfort is more important than lane keeping. Staying in the center of the lane all the time may not be a good option. Most people will drive close to the right lane boundary when on a slope turning right and close to the left lane boundary when on a slope turning left, making the radius larger than just driving along the center, and the forces applied to the driver / passenger are reduced. If this scenario is in the training database, then the trained model should be able to provide a torque curve that is closer to that of a human.

[0056] Thus, a humanized steering system (system 10) for an automated vehicle and a controller 32 for the system 10 are provided. The system 10 uses deep learning techniques so that the steering model 34 can better mimic the steering behavior of a human (eg, operator 14).

[0057] While the present invention has been described according to the preferred embodiments thereof, it is not intended to be limited thereto but only to the scope set forth in the claims that follow.

Claims

1. A controller for a system for generating a steering model, the controller being configured to: For each of a plurality of times the vehicle is being manually steered: receiving parameters at a corresponding time, the parameters comprising a corresponding relative position of a corresponding vehicle relative to at least one object proximate to the corresponding vehicle and a steering angle of the corresponding vehicle at the corresponding time; forming input data indicating a corresponding time of said parameter at said corresponding time; as well as A steering model is constructed based on the input data at the plurality of times, the steering model being configured to: At times when the vehicle is not being manually steered: receiving input data indicating for the time a parameter at the time including a relative position of the vehicle at the time; and A desired steering angle for the time is generated based on the input data for the time.

2. The controller according to claim 1, wherein: The input data comprises a binary vector.

3. The controller according to claim 2, wherein: The binary vector contains bits for each of the parameters.

4. The controller according to claim 1, wherein: The parameters further include weather conditions.

5. The controller according to claim 4, wherein: Said weather conditions include sunny, cloudy, foggy, rainy, snowy, windy and temperature.

6. The controller according to claim 1, wherein: The parameters further include vehicle status.

7. The controller according to claim 6, wherein: The vehicle state includes one or more of lane position, lateral acceleration, longitudinal acceleration, or vehicle speed.

8. The controller according to claim 1, wherein: The parameters further include vehicle specifications.

9. The controller according to claim 8, wherein: The vehicle specifications include one or more of vehicle size, vehicle mass, steering response time, or engine power.

10. The controller according to claim 1, wherein: The parameters further include object characteristics.

11. The controller according to claim 10, wherein: The object characteristics include one or more of an object position, an object vector, an object classification, or a lane position.

12. The controller according to claim 1, wherein: The parameters further include weather conditions, vehicle status, vehicle specifications and object characteristics.

13. The controller according to claim 1, wherein: The construction of the steering model includes applying at least one deep learning technique.

14. The controller according to claim 1, wherein: The steering model includes at least one recurrent neural network; and The construction of the steering model includes training the recursive neural network.

15. The controller of claim 14, wherein: The input data is labeled; and The training of the recurrent neural network is supervised.

16. A controller for a system for steering a vehicle, said The controller is configured to: At times when the vehicle is not being manually steered: receiving parameters at the time, the parameters comprising a relative position of the vehicle relative to at least one object proximate to the vehicle; forming input data for the time indicative of the parameter at the time; feeding the input data into a steering model, wherein the steering model is determined based on at least a steering angle of the vehicle when the vehicle is manually steered and a relative position of the vehicle with respect to at least one object proximate to the vehicle; receiving, from the steering model, a desired steering angle for the time based on the input data for the time; and A steering actuator of the vehicle is caused to steer the vehicle according to the desired steering angle for that time.

17. The controller according to claim 16, wherein: The input data comprises a binary vector containing bits for each of the parameters.

18. A method for generating a steering model, the method using the controller according to any one of claims 1-15 to generate the steering model.

19. A method for steering a vehicle, the method using a controller according to claim 16 or 17 to steer the vehicle.

Images