VEHICLE CONTROL THAT COMBINES A NEURAL NETWORK AND PHYSICAL PREDICTION

Abstract

In exemplary embodiments, methods and systems are provided that include one or more vehicle sensors and a vehicle processor. The one or more sensors are configured to acquire sensor data about the vehicle's operation. The processor is coupled to the one or more sensors and is configured to enable at least the input of the sensor data into a grey-box model, including a neural network model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements; the determination of a vehicle trajectory using the grey-box model; the determination of a control action for the vehicle based on the grey-box model using the sensor data; and the execution of the control action for the vehicle in accordance with the instructions provided by the processor.

Description

INTRODUCTION

[0001] The technical field generally refers to platforms such as vehicles and in particular to methods and systems for controlling vehicles, including the control of brakes, steering and torque based on the vehicle trajectory.

[0002] Many vehicles today use techniques to monitor vehicle trajectory and to control the vehicle based on that trajectory. However, in certain situations, these techniques are not always optimal.

[0003] Accordingly, it is desirable to provide improved methods and systems for controlling vehicles that are also based on vehicle trajectory. Furthermore, other desirable features and characteristics of the present disclosure will become apparent from the following detailed description and the accompanying claims in conjunction with the accompanying drawings and the preceding technical field and background. DESCRIPTION

[0004] According to an exemplary embodiment, a method is provided that includes: obtaining, via one or more sensors of a vehicle, sensor data about the operation of the vehicle; inputting the sensor data into a grey-box model, consisting of a neural network model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements, via a processor of the vehicle; determining, via the processor, a trajectory of the vehicle using the grey-box model; determining, via the processor, a control action for the vehicle based on the grey-box model using the sensor data; and executing the control action for the vehicle in accordance with the instructions provided by the processor.

[0005] In an exemplary embodiment, the control action also includes one or more of the following: applying a steering angle to the vehicle, applying a braking torque to the vehicle, and applying a drive motor torque to the vehicle.

[0006] In an exemplary embodiment, the control action also includes each of the following: applying the steering angle to the vehicle, applying the braking torque to the vehicle, and applying the drive motor torque to the vehicle.

[0007] Also in an exemplary embodiment, the neural network model relates to a vehicle-to-ground interaction between a plurality of wheels of the vehicle and the ground of a roadway on which the vehicle is traveling; and the neural network model is updated based on an application of the physics-based model.

[0008] Also in an exemplary embodiment, the neural network model includes a Jacobian matrix containing a gradient matrix designed to improve the efficiency of the neural network model and thus the control of the vehicle.

[0009] In another exemplary embodiment, the neural network model is trained using the grey-box model together with vehicle simulation data and vehicle test data.

[0010] In an exemplary embodiment, the method further includes developing a model-based control system for the vehicle, which is based on a linear time-varying (LTV) approximation of the grey-box model.

[0011] Also in an exemplary embodiment, the physics-based model is implemented by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and by using outputs of the neural network model and estimates of a longitudinal acceleration, a lateral acceleration and a yaw acceleration of the vehicle using one or more model-based control methods, comprising one or more methods with model predictive control (MPC), methods with linear-quadratic controllers (LQR) or methods with iterative linear-quadratic controllers (iLQR).

[0012] Also in an exemplary embodiment, the physics-based model is implemented by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and by using outputs of the neural network model and estimates of longitudinal acceleration, lateral acceleration and yaw acceleration of the vehicle using one or more model-based control methods, comprising each of the following: one or more model predictive control (MPC) methods, one or more linear quadratic controller (LQR) methods, and one or more iterative linear quadratic controller (iLQR) methods.

[0013] In another exemplary embodiment, a system is provided comprising one or more vehicle sensors and a vehicle processor. The one or more sensors are configured to acquire sensor data about the vehicle's operation. The processor is coupled to the one or more sensors and is configured to enable the input of the sensor data into a grey-box model, which includes a neural network model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements; the determination of a vehicle trajectory using the grey-box model; the determination of a control action for the vehicle based on the grey-box model using the sensor data; and the execution of the control action for the vehicle in accordance with the processor's instructions.

[0014] In an exemplary embodiment, the control action also includes one or more of the following: applying a steering angle to the vehicle, applying a braking torque to the vehicle, and applying a drive motor torque to the vehicle.

[0015] In an exemplary embodiment, the control action also includes each of the following: applying the steering angle to the vehicle, applying the braking torque to the vehicle, and applying the drive motor torque to the vehicle.

[0016] Also in an exemplary embodiment, the neural network model relates to a vehicle-to-ground interaction between a plurality of wheels of the vehicle and the ground of a roadway on which the vehicle is traveling; and the neural network model is updated based on an application of the.

[0017] Also in an exemplary embodiment, the neural network model includes a Jacobian matrix containing a gradient matrix designed to improve the efficiency of the neural network model and thus the control of the vehicle.

[0018] In another exemplary embodiment, the neural network model is trained using the grey-box model together with vehicle simulation data and vehicle test data.

[0019] In an exemplary embodiment, the processor is also configured to enable at least the development of a model-based control system for the vehicle, based on a linear time-varying (LTV) approximation of the grey-box model.

[0020] In an exemplary embodiment, the processor is also configured to enable at least one implementation of the physics-based model by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and using outputs of the neural network model and estimates of a longitudinal acceleration, a lateral acceleration, and a yaw acceleration of the vehicle using one or more model-based control methods, comprising one or more methods with model predictive control (MPC), methods with linear-quadratic controllers (LQR), or methods with iterative linear-quadratic controllers (iLQR).

[0021] Also in an exemplary embodiment, the processor is configured to implement the physics-based model by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and using outputs of the neural network model and estimates of a longitudinal acceleration, a lateral acceleration and a yaw acceleration of the vehicle using one or more model-based control methods, comprising each of the following: one or more model predictive control (MPC) methods, one or more linear-quadratic controller (LQR) methods, and one or more iterative linear-quadratic controller (iLQR) methods.

[0022] In another exemplary embodiment, a vehicle is provided comprising a body, a drive system, a steering system, a braking system, and a processor. The drive system, steering system, and braking system are configured to control the movement of the body. The one or more sensors are configured to obtain sensor data about the operation of the vehicle.The processor is coupled to one or more sensors and is configured to enable the input of sensor data into a grey-box model, which includes a neural network model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements; the determination of a vehicle trajectory using the grey-box model; the determination of a control action for the vehicle based on the grey-box model using the sensor data; and the execution of the control action for the vehicle in accordance with the processor's instructions.

[0023] In another exemplary embodiment, the neural network model relates to the vehicle-ground interaction between a multitude of the vehicle's wheels and the road surface on which the vehicle travels; the neural network model is updated based on the application of the physics-based model; the neural network model includes a Jacobian matrix containing a gradient matrix designed to improve the efficiency of the neural network model and thus the vehicle control; the neural network model is trained using the grey-box model along with vehicle simulation data and vehicle test data; the processor is further trained to enable at least one model-based control for the vehicle based on a linear time-variable (LTV) approximation of the grey-box model;and the processor is further trained, at least to implement the physics-based model by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and using outputs of the neural network model and estimates of a longitudinal acceleration, a lateral acceleration and a yaw acceleration of the vehicle using one or more model-based control methods, comprising each of the following: one or more model predictive control (MPC) methods, one or more linear-quadratic controller (LQR) methods, and one or more iterative linear-quadratic controller (iLQR) methods.; DESCRIPTION OF THE DRAWINGS

[0024] The present disclosure is described below in conjunction with the following drawings, where identical numbers denote identical elements, and where: Fig. 1 a functional block diagram of a vehicle comprising a control system for controlling a vehicle, including on the basis of the vehicle trajectory, using a network model for the forces acting on the vehicle and a physics-based model for body movements of the vehicle, according to exemplary embodiments; Fig. Figure 2 is a flowchart of a procedure for controlling a vehicle, including based on the vehicle trajectory, using a mesh model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements, and which, in conjunction with the vehicle, Fig. 1 can be implemented according to exemplary embodiments; Fig. Figure 3 is an illustration of a subprocess of the process of Fig. 2, including implementation in a vehicle during an ongoing vehicle propulsion system, according to exemplary embodiments; Fig. Figure 4 is an illustration of an exemplary neural network that, in conjunction with the method of Fig. 2 can be implemented in accordance with exemplary embodiments; Fig. Figure 5 is an illustration of an exemplary implementation of one step of the procedure of Fig. 2, namely the development of a physics-based model for the body movements of the vehicle, according to exemplary embodiments; Fig. Figure 6 is an illustration of an exemplary implementation of one step of the procedure of Fig. 2, namely, training a neural network model for the forces acting on the vehicle, according to exemplary embodiments; and Fig. Figure 7 is an illustration of an exemplary implementation of a substep of the step of Fig. 6 and the procedure of Fig. 2, namely the training and testing of the neural network model. DETAILED DESCRIPTION

[0025] The following detailed description is merely exemplary and is not intended to limit the disclosure or the applications and uses thereof. Furthermore, there is no intention to be bound by the theory presented in the preceding background or in the following detailed description.

[0026] Fig. Figure 1 shows a vehicle 100 according to an exemplary embodiment. As described in more detail below, the vehicle 100 comprises, among other components, a control system 102 for controlling a vehicle, including on the basis of the vehicle trajectory, using a neural network and a physics-based model for the vehicle's body movements, according to exemplary embodiments. As described below in connection with Fig. 1 and the procedure 200 of Fig. 2 and the implementations of Fig. As described in more detail in Figures 3-7, the control system 102 uses a neural network and a physics-based model for vehicle body movements in controlling braking, steering, torque and other aspects of vehicle motion and control actions based on the trajectory, including a non-linear trajectory of the vehicle 100 under various circumstances.

[0027] In various embodiments, the vehicle 100 comprises an automobile, such as any number of different types of automobiles, like a sedan, a station wagon, a truck, a sport utility vehicle (SUV), or the like. In certain embodiments, the vehicle 100 may also comprise a motorcycle or another vehicle, such as an aircraft, a spacecraft, a watercraft, etc., and / or one or more other types of mobile platforms (e.g., a robot and / or another mobile platform).

[0028] In the illustrated embodiment, the vehicle 100 comprises a body 104 mounted on a chassis 116. The body 104 essentially encloses other components of the vehicle 100. The body 104 and the chassis 116 can together form a frame. The vehicle 100 also comprises a plurality of wheels 112. The wheels 112 are each rotatably connected to the chassis 116 near a corner of the body 104 to enable the movement of the vehicle 100. In one embodiment, the vehicle 100 comprises four wheels 112, although this may vary in other embodiments (e.g., in trucks, motorcycles, and certain other vehicles).

[0029] A drive system 110 is mounted on the chassis 116 and drives the wheels 112, for example via axles 114. In certain embodiments, the drive system 110 comprises a drive system with a motor 113 (which, for example, in various embodiments comprises one or more internal combustion engines, electric motors, or the like).

[0030] As in Fig. As shown in Figure 1, the vehicle in various embodiments also includes a braking system 106 and a steering system 108. In exemplary embodiments, the braking system 106 controls the braking of the vehicle 100 using brake components that are controlled by inputs from a driver (e.g. via a brake pedal 107) and / or automatically via a control system (such as the control system 102 and / or one or more other control systems).

[0031] In exemplary embodiments, the steering system 108 also controls the steering of the vehicle 100 via steering components that are controlled by inputs from a driver (e.g. via a steering wheel 109) and / or automatically via a control system (such as the control system 102 and / or one or more other control systems).

[0032] In the Fig. In the embodiment shown in Figure 1, the control system 102 is connected to the braking system 106, the steering system 108, and the drive system 110 and controls their operation and functionality. In various embodiments, the control system 102 also controls the vehicle using a neural network and a physics-based model for the vehicle's body movements, as described in Figure 1. Fig. 2 the procedures shown 200 and the implementations of the Fig. 3-7 and as described below in connection therewith.

[0033] As also in Fig. As shown in Figure 1, the control system 102 in various embodiments comprises a sensor arrangement 120, a display 130, a tracking system 136 and a control unit 140, as described in more detail below.

[0034] In various embodiments, the sensor array 120 comprises different sensors that obtain sensor data about the operation of the vehicle 100 and its trajectory. In various embodiments, the sensor array 120 comprises different IMU (Inertial Measurement Unit) sensors 122 (e.g., various speed sensors, accelerometers, gyroscopes, torque sensors, steering angle and / or wheel angle sensors, etc.). In certain embodiments, the sensor array 120 may also include one or more other sensors 124, such as one or more input sensors and / or one or more other sensors that measure other vehicle and / or environmental parameters.

[0035] In various embodiments, the display 130 provides information and instructions for the occupants of the vehicle 100 (including, in various embodiments, a driver and other occupants of the vehicle 100), among other content. As in Fig. As shown in Figure 1, the display 130 in various embodiments comprises an audio component 132 (including one or more loudspeakers) for displaying audio instructions and other information and content for the passengers, in addition to a visual (or video) component 134 (including one or more screens) for displaying visual instructions and other information and content for the passengers.

[0036] In various embodiments, the tracking system 136 also obtains information about the geographic location and position of the vehicle 100. In certain embodiments, the tracking system 136 includes a navigation system for the vehicle 100. In certain embodiments, the tracking system 136 also includes a satellite-based tracking system, such as a global positioning system (GPS) and / or another satellite-based system.

[0037] In various embodiments, the control unit 140 is connected to the sensor arrangement 120, the display 130, and the location system 136. In various embodiments, the control unit 140 receives sensor data from the sensor arrangement 120 (and in certain embodiments also location data from the location system 136), interprets and processes the sensor data (and in certain embodiments also the location data), and outputs instructions and other information and content based thereon via the display 130. In various embodiments, the control unit 140 also controls various actions of the vehicle (e.g., braking, steering, drive torque, etc.), among other things, based on a calculated trajectory of the vehicle 100 based on the sensor data (and in certain embodiments also based on the location data).In various embodiments, the control unit 140 is also connected to the brake system 106, the steering system 108 and the drive system 110, as well as to various other vehicle components (e.g. including a navigation system and other components not shown) and controls their operation.

[0038] In various embodiments, the control unit 140 provides these functions in accordance with the steps of process 200, which is described in Fig. 2 is shown and further below in connection with it and in connection with the implementations of the Fig. 3-7, which are also described in more detail below, will be explained in more detail.

[0039] As in Fig. As shown in Figure 1, the control unit 140 in various embodiments comprises a computer system (here also referred to as computer system 140) which includes a processor 142, a memory 144, an interface 146, a storage device 148 and a computer bus 150.

[0040] The processor 142 performs the calculation and control functions of the control unit 140 and can comprise any type of processor or multiple processors, individual integrated circuits such as a microprocessor, or any number of integrated circuits and / or printed circuit boards working together to perform the functions of a processing unit. During operation, the processor 142 executes one or more programs 152 contained in the memory 144 and, as such, controls the general operation of the control unit 140 and the computer system of the control unit 140, generally in the execution of the processes described herein, such as process 200 of Fig. 2 and the implementations of Fig. 3-7 and as described below in connection therewith.

[0041] The memory 144 can be any suitable type of memory, including various types of non-transferable, computer-readable storage media. In certain examples, the memory 144 is located on the same computer chip as the processor 142 and / or is arranged on it. In the illustrated embodiment, the memory 144 stores the aforementioned program 152 together with a neural network model 154 and stored values ​​157 (e.g., lookup tables, thresholds, and / or other values ​​relating to the process 200).

[0042] Interface 146 enables communication with the computer system of the control unit 140, for example, from a system driver and / or another computer system, and can be implemented using any suitable method and device. In one embodiment, interface 146 links the various data from the sensor array 120 with other possible data sources. Interface 146 can include one or more network interfaces for communication with other systems or components. Interface 146 can also include one or more network interfaces for communication with technicians and / or one or more memory interfaces for connecting to storage devices, such as the storage device 148.

[0043] The storage device 148 can be any suitable type of storage device, including various types of random-access memory and / or other storage devices. In an exemplary embodiment, the storage device 148 comprises a program product from which the memory 144 can receive a program 152 that executes one or more embodiments of one or more processes of the present disclosure, such as the steps of process 200 of Fig. 2 and implementations of the Fig. 3-7 and as described below in connection therewith. In another exemplary embodiment, the program product can be stored directly in memory 144 and / or on a disk (e.g. disk 156), as described below, and / or accessed in another way.

[0044] Bus 150 is used to transmit programs, data, status, and other information or signals between the various components of the computer system of the control unit 140. Bus 150 can be any suitable physical or logical means of connecting computer systems and components. This includes, but is not limited to, direct, hard-wired connections, fiber optic technology, infrared, and wireless bus technologies. During operation, the program 152 is stored in memory 144 and executed by processor 142.

[0045] While this exemplary embodiment is described in connection with a fully functional computer system, the person skilled in the art will recognize that the mechanisms of the present disclosure can be distributed as a program product with one or more types of non-transitory, computer-readable, signal-carrying media used to store the program and its instructions and to carry out its distribution, such as a non-transitory, computer-readable medium carrying the program and containing computer instructions stored therein to cause a computer processor (such as processor 142) to carry out and execute the program.

[0046] Fig. Figure 2 is a flowchart of a process 200 for controlling a vehicle, including on the basis of the vehicle trajectory, using a neural network and a physics-based model for the vehicle's body movements, according to an exemplary embodiment. In various embodiments, the process 200 can be used in conjunction with the vehicle 100. Fig. 1, including its control system 102, will be implemented. The procedure is also described below in connection with the Fig. 3-7 describe, which show exemplary representations of certain steps of procedure 200.

[0047] As in Fig. As shown in Figure 2, process 200 begins at 202 in various embodiments. In various embodiments, process 200 begins, at least in a first iteration, before the vehicle 100 is driven by the user in an actual vehicle drive. In various embodiments, process 200 also begins with a first subprocess 203 (which comprises steps 204-208, as shown in Figure 2). Fig. 2 shown and described below), before the current vehicle is driven.

[0048] In various embodiments, a neural network model for the forces acting on the vehicle is designed (step 204). In various embodiments, the neural network model is implemented via a processor (such as processor 142 from Fig. 1) designed with regard to vehicle-ground interaction, including the interaction between the wheels 112 of the vehicle 100 of Fig. 1 (or tires coupled to it) and the surface of the roadway or path on which the vehicle travels at 100. In various embodiments, the design of the neural network model also includes the definition of a neural network structure and a size for the application to the neural network model. An exemplary neural network model is in Fig. 4 is shown and is described in more detail below.

[0049] In various embodiments, the development of the network model for the forces acting on the vehicle (and the physically based model for the vehicle's body motion, described below) is used to calculate the trajectory and trajectory rate for vehicle 100 for various situations, including situations with extreme maneuvers involving high lateral accelerations and / or yaw rates, and including situations where the vehicle's behavior is close to or approaching the applicable handling limits, and / or where the handling and / or trajectory of vehicle 100 is non-linear.

[0050] In various embodiments, a physics-based model for the vehicle's body movements is developed (step 206). In various embodiments, the physics-based model is implemented via a processor (such as the Processor 142 from Fig. 1) developed. In various embodiments, the physics-based model takes the outputs of the neural network model and estimates the longitudinal, lateral, and yaw accelerations of the vehicle. In various embodiments, the hybrid model (i.e., the use of the neural network model and the physics-based model) allows the use of model-based control methods, such as Model Predictive Control (MPC), Iterative Quadratic Regular (LQR), and Iterative Linear Quadratic Regulator (iLQR). An exemplary implementation of the development of a physics-based model is shown in Fig. 5 is shown and is described in more detail below.

[0051] In various embodiments, training is performed on the neural network model (step 207). In various embodiments, the neural network model is trained using a grey-box model that combines the physical model with the neural network model. Also in various embodiments, the training of the neural network model is performed by processor 142. Fig. 1. Using the grey-box model together with simulation data for vehicle 100 together with vehicle test data. Also in various embodiments, the neural network model is validated by evaluating its performance with test data and through simulation or in-vehicle testing under different driving scenarios (e.g., using different vehicle parameters and different vehicle environment conditions, etc.). An exemplary implementation of the neural network model training is shown in Fig. 6 is shown and is described in more detail below in connection with this.

[0052] In various embodiments, a model-based control system for the vehicle is developed (step 216). In various embodiments, the processor 142 develops Fig. 1. The model-based control system uses linear time evolution (LTV) as an approximation of the grey-box model described above. In various embodiments, the model-based control system includes both (1) an optimization algorithm and (2) a model of the vehicle 100.

[0053] As in Fig. As shown in Figure 2, process 200 in various embodiments is continued with a second subprocess 203 (which includes steps 210-220 described below) during the current journey of the vehicle, in which the neural network model, the physics-based model and the resulting grey-box model and model-based control are used to control vehicle actions, including steering, braking and engine torque.

[0054] In various embodiments, sensor data is obtained (step 210). In various embodiments, the sensor data includes inertial measurement sensor data from the IMU 122. Fig. 1. In various embodiments, the sensor data includes accelerations, velocities, angular velocities and coordinates of the vehicle 100.

[0055] In various embodiments, the sensor data are also applied to the model (step 212). In various embodiments, the sensor data are applied to the grey-box model described above, which is a hybrid model combining the neural network model and the physics-based model described above. In various embodiments, this is done via processor 142. Fig. 1 is carried out and results in output values ​​of the initial state (x) of vehicle 100 together with ground forces (F) between vehicle 100 and the ground of the roadway on which vehicle 100 is traveling.

[0056] In various embodiments, a reference trajectory is obtained (step 214), along with constraints for vehicle 100 (step 215). In various embodiments, these values ​​are provided by processor 142. Fig. 1 received on the basis of previous data values ​​(e.g. from a previous iteration of process 200) and / or stored values ​​157, which are stored in memory 144 for vehicle 100.

[0057] In various embodiments, the outputs of the model from step 212, together with the reference trajectory from step 214 and the constraints from step 215, are provided to the model predictive control (MPC) for processing (step 216).

[0058] In various embodiments, the MPC uses an initial state of the vehicle (e.g., using the physics-based model) and optimizes the performance criteria for the neural network model for a predetermined number of steps. In other embodiments, the MPC starts with the initial state of the vehicle (e.g., in an initialization mode) and performs efficient model optimization at each timestamp (thus optimizing the performance criteria). Similarly, in various embodiments where the MPC is performed cyclically, a determination of the new state of the vehicle is carried out. In these embodiments, the new state of the vehicle is then used as the "initial state" in the next iteration of the model, and the model is then executed from there using an optimization algorithm to optimize the performance criteria.In various embodiments, the first step of the vehicle's control sequence is then applied, and the process continues. In these embodiments, this sequence corresponds to a "receding horizon" control system, where the model optimizes a predefined horizon and is then updated at each step and carried forward for the various subsequent steps, and so on. In various embodiments, this enables efficient creation of the optimization criteria and efficient use of computing and other resources.

[0059] The MPC also uses the 142 processor in various embodiments. Fig. 1. The various inputs from steps 212, 214, and 215 are used to determine control commands for vehicle 100 based on the trajectory of vehicle 100 (step 218). In various embodiments, the control commands include braking, steering, and drive torque commands for vehicle 100 based on the vehicle trajectory.

[0060] The control commands are also implemented in various embodiments (step 220). In particular, in various embodiments, the control commands from step 218 are implemented in step 220 via the applicable vehicle systems (such as the braking system 106, the steering system 108, and the drive system 110). Fig. 1) implemented in instructions executed by processor 142 based on the provisions of process 200 of Fig. 2, including the provisions of steps 216 and 218, will be provided.

[0061] In various embodiments, process 200 is then terminated in step 222.

[0062] With reference to Fig. 3 will be a representation of subprocess 209 of process 200 of Fig. 3, including the implementation in vehicle 100 during an ongoing vehicle journey, according to exemplary embodiments. As shown in Fig. As shown in Figure 3, in an exemplary embodiment the sensor data are collected in step 210 (e.g. from the IMU 122 of Fig. 1) and the grey-box model in step 212 (in which values ​​are determined, including for an initial state (X) for the vehicle and forces (F) of the interaction between the wheels 112 of the vehicle 100 and the road surface on which the vehicle 100 is traveling. As also in Fig. As shown in Figure 3, a reference trajectory and vehicle restrictions are obtained (steps 214 and 215, respectively), and these values, along with the grey-box model determinations from step 212, are entered into the MPC (step 216). As shown in Fig. As shown in Figure 3, the provisions of the MPC from Step 216 are used in determining and providing control actions for Vehicle 100 (Steps 218 and 220), including steering angle, engine torque, and braking torques for Vehicle 100. As also shown in Figure 3, the provisions of the MPC from Step 216 are used in determining and providing control actions for Vehicle 100 (Steps 218 and 220), including steering angle, engine torque, and braking torque for Vehicle 100. Fig. As shown in Figure 3, in certain embodiments the actions of steps 218 and 220 are then used to generate new and / or updated sensor data in a new iteration of step 210 when the process is repeated in a new iteration, and so on. In certain embodiments, information and / or indications of the control action for the driver and / or other occupants of the vehicle 100 are also displayed on the display 130. Fig. 1 provided (e.g. including acoustic, visual, haptic and / or other notifications of the control action), in accordance with instructions provided by processor 142.

[0063] Fig. Figure 4 is an illustration of an exemplary neural network model 154, which is used in conjunction with the procedure 200 of Fig. 2 can be implemented in accordance with exemplary embodiments. As in Fig. As shown in Figure 4, the neural network model 154 comprises at least four layers in various embodiments, including an input layer 402, several internal processing layers 404, and an output layer 406. In various embodiments, the input layer 402 includes input characteristics such as driving and braking torques, normal loads, slip angles, chassis speeds, and yaw rate (e.g., as obtained via the IMU 122 of Fig. 1 determined). Also in various embodiments, the neural network model 154 comprises a network architecture with four neural network mappings. Furthermore, in various embodiments, the neural network model 154 comprises a network configuration that includes various layers and neurons, as well as an activation function and a training algorithm. In various embodiments, the neural network model 154 comprises a data-driven model of the vehicle-ground interaction and is stored in the memory 144 of the vehicle 100, as described in Fig. 1 shown. In various embodiments, the neural network model 154 also includes a Jacobian matrix (i.e., a gradient matrix) to improve the efficiency of the neural network model 154 and thus the control of the vehicle 100.

[0064] In certain embodiments, the Neural Network Model 154 also includes a data-driven model of vehicle-ground interaction, which is represented by the following equation (Equation 1): (FxiFyi)=fnni(Ti,vx,vy,r,αi,Fzi;wi), for i=1,2,3,4

[0065] In the fnni Represents neural network models.

[0066] Fig. Figure 5 is an illustration of an exemplary implementation of step 206 of procedure 200 of Fig. 2, namely the development of a physics-based model for the body movements of the vehicle, according to exemplary embodiments. In various embodiments, Newtonian principles are applied to the chassis dynamics of the vehicle 100. Also in various embodiments, trigonometric equations are used for the chassis kinematics of the vehicle 100. Furthermore, in various embodiments, the physics-based model can be extended as part of step 206 by adding actuator models to the physics-based model.

[0067] In various embodiments, such as in Fig. As shown in Figure 5, in various embodiments, as part of step 206, a force is exerted on each corner of the vehicle 100, namely on the respective wheels 112 at each corner of the vehicle 100, which interact with the ground of the roadway on which the vehicle 100 travels.

[0068] As in Fig. Figure 5 shows various embodiments in which an X-axis 501 and a Y-axis 502 are depicted transversely across the vehicle 100 (e.g., with the X-axis 501 extending transversely across the vehicle 100 in a first direction from the rear to the front of the vehicle 100, and with the Y-axis 502 extending transversely across the vehicle 100 in a second direction, a vertical direction, from the passenger side to the driver's side of the vehicle 100). As also shown in Fig. As shown in section 5, a first length (or front length) is 503 (l f ) between a front wheel 112 and a center of the vehicle 100 is shown, while a second length (or rear length) 504 (l r ) between a rear wheel 112 and the center of the vehicle 100. As also shown in Fig. Figure 5 shows a steering angle β 504 for vehicle 100, together with corresponding speeds, which represent a first speed V x507 along the X-axis 501, a second velocity V y along the Y-axis 502 and a directional velocity V 509 along the steering angle β 504.

[0069] As in Fig. As shown in Figure 5, a first torque is applied to a front wheel on the driver's side 112, with first torque components Fx1 510 along the X-axis 501 and F Y1 512 along the Y-axis 502, and with a force angle δ 512. As also in Fig. As shown in Figure 5, a second torque is applied to a wheel on the passenger side 112, with second torque components Fx2 513 being applied along the X-axis 501 and FY2 514 along the Y-axis 502, at the same force angle as the front wheel on the driver's side in an exemplary embodiment. Furthermore, as also shown in Figure 5, a second torque component is applied to a wheel on the passenger side 112, with second torque components Fx2 513 being applied along the X-axis 501 and FY2 514 along the Y-axis 502, at the same force angle as the front wheel on the driver's side in an exemplary embodiment. Fig. Figure 5 shows a third torque being applied to a rear wheel on the driver's side 112, wherein third torque components Fx3 515 are applied along the X-axis 501 and FY3 516 are applied along the Y-axis 502, while a fourth torque is applied to a rear wheel on the passenger's side 112, wherein fourth torque components F x4 517 along the X-axis 501 and FY4 518 along the Y-axis 502.

[0070] In certain embodiments, the physical model includes the following equations: v˙x=ax+rvy=Fx1 cos δf−Fy1 sin δf+Fx2 cos δf−Fy2 sin δf+Fx3+Fx4m+rvy v˙y=ay+rvx=Fy1 cos δf−Fx1 sin δf+Fy2 cos δf−Fx2 sin δf+Fy3+Fy4m+rvx r˙=−lwr(Fx3−Fx4)+lf(Fy1 cos δf+Fy2 cos δf+Fx1 sin δf+Fx2 sin δf)−lr(Fy3+Fy4)+lwf(−Fx1 cos δf+Fx2 cos δf+Fy1 sin δf−Fy2 sin δf)Izz X˙=vx cos ψ−vy sin ψ Y˙=vx sin ψ+vy cos ψ ψ˙=r where "F" stands for force, "v" for velocity, "a" for acceleration, and the other designations in Fig. 5 and / or in connection with the above discussion.

[0071] In various embodiments, the neural network model (e.g., as in conjunction with the implementation of Fig. 5 developed and as in Fig. 4 shown and described in connection therewith) the complicated tire model required for the physics-based model, which increases model fidelity and reduces computational complexity.

[0072] With reference to Fig. Section 6 will be an exemplary implementation of step 207 of the procedure of Fig. 2, namely the training of a neural network model for the forces acting on the vehicle, is illustrated according to exemplary embodiments. In various embodiments, step 207 begins at 602, as shown in Fig. 6 shown.

[0073] As in Fig. As shown in Figure 6, simulation scenarios for the neural network model are defined in various embodiments (step 604). In various embodiments, processor 142 defines Fig. 1. The simulation scenarios are designed to cover various possible driving scenarios for the vehicle, e.g., also with regard to different vehicle parameters and environmental conditions. In various embodiments, the set of scenarios is extensive (with many scenarios) to ensure adequate coverage of the required operating range for the vehicle 100. In various embodiments, the simulation scenarios are also defined to include various vehicle and environmental parameters and their ranges of change.

[0074] As in Fig. As shown in Figure 6, scenario simulations are performed in various embodiments (step 606). In various embodiments, the processor 142 of Fig. 1. The scenario simulations are performed to include a broad and extensive range of vehicle and environmental parameters, generating sufficient offline training data for the vehicle 100. In various embodiments, this helps to ensure a high level of steady-state accuracy for the neural network model.

[0075] As in Fig. As shown in Figure 6, the neural network model is trained and tested in various embodiments (step 608). In various embodiments, the training and testing is performed by processor 142. Fig. 1 using data collected during the simulation.

[0076] In various embodiments, the training of the neural network model is performed using a loss function according to the following equations: ax=1m∑i=14c1i(δf)Tfnni(Ti,vx,vy,r,αi,Fzi;wi) ay=1m∑i=14c2i(δf)Tfnni(Ti,vx,vy,r,αi,Fzi;wi) r˙=1Izz∑i=14c3i(δf)Tfnni(Ti,vx,vy,r,αi,Fzi;wi) where “a x “ and “a y “represent the acceleration in the “x” or “y” direction and the “c” values ​​are suitable parameters.

[0077] In certain embodiments, the training loss function can also be expressed in a vector format, according to the following equation: (axayr˙)︸y=(c11(δf)Tmc12(δf)Tmc13(δf)Tmc14(δf)Tmc21(δf)Tmc22(δ f)Tmc23(δf)Tmc24(δf)Tmc31(δf)TIzzc32(δf)TIzzc33(δf)TIzzc34(δf)T Izz)︸M(δf)⋅(fnn1(T1,vx,vy,r,α1,Fz1;w1)fnn2(T2,vx,vy,r,α2,Fz2;w2 )fnn3(T3,vx,vy,r,α3,Fz3;w3)fnn4(T4,vx,vy,r,α4,Fz4;w4))︸Fnn(p;w)

[0078] in which: "p" is the vector of all network inputs: T1, T2, T3, T4, V x , V y , r, a1, a2, a3, a4, F z1 , F z2 , F z3 , and F z4 ; and "w" is the vector of all network weights: w1, w2, w3, and w4.

[0079] In various embodiments, the loss function can also be optimized with respect to "w" according to the following equation: 1N∑i=1N‖ym−y‖2+λ2‖w‖2=1N∑i=1N‖ym−M(δfm)Fnn(pm;w)‖2+λ2‖w‖2

[0080] With reference to Fig. 7 includes the training and testing of step 608 in various embodiments, the testing of the vehicle 702, which generates training data 704 for the neural network model. As also described in Fig. As shown in Figure 7, in various embodiments, further training and test data 706 are used together with the aforementioned training data 704 from the vehicle tests 702 to perform offline training 708 for the neural network model. In various embodiments, the offline training 708 then results in an updated and trained neural network model 712.

[0081] Returning to Fig. In various embodiments, further training of the neural network model is performed (step 610). In various embodiments, the processor 142 of Fig. 1. Further training is performed on the updated and trained neural network model 712, among other things to further train the neural network model for accuracy using data collected during the vehicle test. In certain embodiments, step 207 then ends at 612, as in Fig. 6 shown.

[0082] Accordingly, procedures, systems, and vehicles are provided for controlling vehicle actions. As in Fig. As shown in Figures 1-7 and described above in connection therewith, the disclosed systems in various embodiments implement a grey-box model that combines a neural network model for the forces acting on the vehicle with a physics-based model for the body movements of the vehicle to determine and execute vehicle control actions, including steering angle, braking torque and engine drive torque, based on the trajectories (including nonlinear trajectories) of the vehicle 100.

[0083] It becomes clear that the systems, vehicles, and procedures may differ from those depicted in the figures and described here. For example, vehicle 100 may differ from those shown in the figures. Fig. 1, including the control system 102 and / or other components thereof, in various embodiments of the one described in Fig. The system shown in Figure 1 and / or described above in connection with it differs. It also becomes clear that the steps of Process 200 and their implementations differ from those in Figure 1. Fig. 2-7 shown can be distinguished and / or that different steps of the process 200 occur simultaneously and / or in a different order than that shown Fig. The sequence shown in 2-7 and / or described above in connection with it can be carried out.

[0084] Although at least one exemplary embodiment has been presented in the foregoing detailed description, it should be understood that a large number of variations exist. It should also be appreciated that the exemplary embodiment or embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of the disclosure in any way. Rather, the foregoing detailed description is intended to provide the person skilled in the art with a practical guide for implementing the exemplary embodiment or embodiments. It is understood that various modifications in the function and arrangement of the elements may be made without departing from the scope of the disclosure as set forth in the appended claims and their statutory equivalents.

Claims

[1] A procedure encompassing: Obtaining sensor data about the operation of the vehicle via one or more sensors of a vehicle; Inputting the sensor data into a grey-box model, consisting of a neural network model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements, via a processor in the vehicle; Determine, via the processor, a trajectory of the vehicle using the grey-box model; Determine, via the processor, a control action for the vehicle, based on the grey-box model using sensor data; and Executing the vehicle's control action in accordance with the instructions provided by the processor. [2] Method according to claim 1, wherein the control action comprises one or more of the following: applying a steering angle to the vehicle, applying a braking torque to the vehicle, and applying a drive motor torque to the vehicle. [3] Method according to claim 2, wherein the control action comprises each of the following: Applying the steering angle to the vehicle, applying the braking torque to the vehicle, and applying the drive motor torque to the vehicle. [4] The method according to claim 1, wherein: The neural network model concerns a vehicle-to-ground interaction between a plurality of wheels of the vehicle and the ground of a roadway on which the vehicle is traveling; and the neural network model is updated based on an application of the physics-based model. [5] Method according to claim 4, wherein the neural network model includes a Jacobian matrix comprising a gradient matrix designed to improve the efficiency of the neural network model and thus the control of the vehicle. [6] Method according to claim 4, wherein the neural network model is trained using the grey box model together with vehicle simulation data and vehicle test data. [7] Method according to claim 6, wherein the method further comprises developing a model-based control system for the vehicle based on a linear time-variable (LTV) approximation of the grey-box model. [8] Method according to claim 4, wherein the physics-based model is implemented by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and using outputs of the neural network model and estimates of a longitudinal acceleration, a lateral acceleration and a yaw acceleration of the vehicle using one or more model-based control methods, comprising one or more model predictive control (MPC) methods, linear quadratic controller (LQR) methods or iterative linear quadratic controller (iLQR) methods. [9] Method according to claim 4, wherein the physics-based model is implemented by applying a torque to each of the four corners of the vehicle, by applying the torque to respective wheels at each of the respective corners of the vehicle, and using outputs of the neural network model and estimates of a longitudinal acceleration, a lateral acceleration and a yaw acceleration of the vehicle using one or more model-based control methods, comprising each of the following: one or more model predictive control (MPC) methods, one or more linear quadratic controller (LQR) methods, and one or more iterative linear quadratic controller (iLQR) methods. [10] A system encompassing: one or more sensors of a vehicle that are designed to obtain sensor data about the operation of the vehicle; and a processor of the vehicle, wherein the processor is coupled and configured with one or more sensors, at least to enable: Inputting the sensor data into a grey-box model that includes a neural network model for the forces acting on the vehicle and a physics-based model for the vehicle's body movements; Determining a vehicle trajectory using the grey box model; Determining a control action for the vehicle, based on the grey-box model using sensor data; and Executing the control action for the vehicle, in accordance with the instructions of the processor.

Images