Intelligent vehicle motion control method and device based on physical information neural network

Abstract

The invention provides an intelligent vehicle motion control method and device based on a physical information neural network, and the method comprises the steps: collecting the control and state data of a virtual vehicle in a simulation environment, and constructing a first data set; collecting control and state data of the intelligent vehicle in a real environment, and constructing a second data set; inputting the first data set into a constructed vehicle motion prediction model for pre-training, and finely adjusting the vehicle motion prediction model by using the second data set to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer and is used for predicting a vehicle state quantity and a vehicle control quantity at a future time point; and integrating the trained vehicle motion prediction model into a motion controller of the intelligent vehicle, and controlling the intelligent vehicle to run by using the motion controller. According to the method, the environmental adaptability is higher, the optimization control result is more accurate, and the tracking precision, the driving stability and the safety of the vehicle are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to an intelligent vehicle motion control method and device based on physical information neural network. Background Art

[0002] Existing intelligent vehicle motion control methods perform well under non-extreme conditions in structured environments, but have large tracking errors and poor vehicle stability in harsh environments or extreme conditions.

[0003] For example, when a vehicle is traveling on a rugged off-road surface or making a large angle turn at high speed, load transfer will occur between the left and right tires and the front and rear tires, and the vehicle's suspension will be greatly compressed or extended. If the ground adhesion coefficient is low, the vehicle will also slip significantly.

[0004] In these cases, the low-degree-of-freedom linear physical models used for vehicle motion control have large errors, while the higher-accuracy multi-degree-of-freedom nonlinear physical models are often difficult to solve and cannot meet the real-time requirements in control.

[0005] In addition, data-driven control methods show greater flexibility and applicability in expressing the dynamic characteristics of the system, so data-driven control methods are being widely studied. Data-driven prediction methods mainly include: artificial neural networks, regression analysis, Bayesian networks, physical information neural networks, etc., all of which use historical feature parameters to train data models. These methods can capture the dynamics of time changes, but lack interpretability, and cannot guarantee that the corrected predictions are physically meaningful. They may produce state predictions that the vehicle cannot physically achieve, and it is difficult to guarantee the accuracy and real-time performance of vehicle control when facing unstructured environments, and the stability and safety of the vehicle cannot be guaranteed. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention proposes a method and device for intelligent vehicle motion control based on a physical information neural network, which can enable the intelligent vehicle to drive accurately and stably under different working conditions.

[0007] On the one hand, the present invention provides an intelligent vehicle motion control method based on a physical information neural network, comprising: collecting control and state data of a virtual vehicle in a simulation environment to construct a first data set; collecting control and state data of an intelligent vehicle in a real environment to construct a second data set; inputting the first data set into a constructed vehicle motion prediction model for pre-training, and using the second data set to fine-tune the vehicle motion prediction model to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer to predict vehicle state quantities and vehicle control quantities at future points; integrating the trained vehicle motion prediction model into a motion controller of an intelligent vehicle, and combining the real-time acquisition of the control and state data of the intelligent vehicle, using the motion controller to control the driving of the intelligent vehicle.

[0008] In one embodiment of the present invention, the vehicle motion prediction model is composed of an input layer, a fully connected layer, an output layer and a physical information layer; wherein the input layer, the fully connected layer and the output layer constitute a deep neural network; wherein the vehicle state quantity and the vehicle control quantity at the current time point and the historical time point are obtained in the input layer, wherein the vehicle state quantity includes the vehicle motion state and the tire force state; the vehicle control quantity includes the front wheel steering angle and acceleration; the vehicle state quantity and the vehicle control quantity are received in the fully connected layer, and after linear transformation and nonlinear transformation, the features related to the vehicle state quantity and the features related to the vehicle control quantity are extracted; the features related to the vehicle state quantity and the features related to the vehicle control quantity are received in the output layer, and the vehicle state quantity and the vehicle control quantity at the future time point are predicted; and the four-wheel vehicle dynamics ordinary differential equations based on the four-wheel vehicle model are configured in the physical information layer, including the longitudinal force-motion balance equation, the lateral force-motion balance equation and the torque-rotation balance equation.

[0009] In one embodiment of the present invention, the output layer includes a force layer, an inertia layer and a motion layer; wherein the force layer predicts the tire force state at a future point in time; the inertia layer predicts the moment of inertia at a future point in time; and the motion layer predicts the center of mass motion state and the vehicle motion state at a future point in time.

[0010] In one embodiment of the present invention, the first loss function of the deep neural network is determined based on the root mean square error of the vehicle state quantity and the vehicle control quantity at the predicted future time point; the vehicle state quantity and the vehicle control quantity at the predicted future time point, as well as the vehicle steering angle and the vehicle mass are input into the four-wheel vehicle dynamics ordinary differential equation to obtain three residuals; the root mean square error of the three residuals is taken to obtain the second loss function of the physical information layer; and the total loss function of the vehicle motion prediction model is determined based on the weighted sum of the first loss function and the second loss function.

[0011] In one embodiment of the present invention, the motion controller includes a ground adhesion coefficient identification module, a state observation module, a reference path module, and a transverse and longitudinal coupling MPC module, wherein the transverse and longitudinal coupling MPC module includes a vehicle motion prediction model, a constraint condition submodule, and an optimal solution submodule; wherein: the ground adhesion coefficient identification module identifies the adhesion coefficient between the vehicle and the ground at the current time point or the next time point; the constraint condition submodule receives the adhesion coefficient to construct a constraint condition; the state observation module generates the vehicle state quantity and vehicle control quantity at the current time point and the historical time point based on the vehicle control and state data obtained from the on-board sensors of the intelligent vehicle in real time; the vehicle state quantity and vehicle control quantity at the current time point and the historical time point are received in the vehicle motion prediction model, and the vehicle state quantity and vehicle control quantity at the future time point are predicted; the reference path module generates a reference path for the vehicle to travel; the optimal solution submodule constructs a cost function based on the reference path and the vehicle state quantity and vehicle control quantity at the future time point, and solves the cost function based on the constraint condition.

[0012] In one embodiment of the present invention, a tracking cost term is constructed based on the error between the predicted vehicle position at a future time point and the corresponding waypoint of the reference path, and the vehicle motion state includes the vehicle position; a driving cost term is constructed based on the change in acceleration and the change in the front wheel steering angle within a given time period; and the cost function is obtained based on the tracking cost term and the driving cost term.

[0013] In one embodiment of the present invention, the motion controller further comprises a yaw moment stabilization control module; the yaw moment stabilization control module receives the predicted vehicle state quantity and vehicle control quantity at a future time point and performs predictive yaw moment control.

[0014] In another aspect, the present invention further provides an intelligent vehicle motion control device based on a physical information neural network, comprising:

[0015] A computing unit, configured to construct a first data set using control and state data of a virtual vehicle in a simulation environment; construct a second data set using control and state data of an intelligent vehicle in a real environment; and,

[0016] The first data set is input into a constructed vehicle motion prediction model for pre-training, and the vehicle motion prediction model is fine-tuned using the second data set to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer for predicting a vehicle state quantity and a vehicle control quantity at a future time point;

[0017] The steering and driving unit is equipped with a motion controller which integrates a trained vehicle motion prediction model; the steering and driving unit is used to acquire control and status data of the intelligent vehicle in real time and use the motion controller to control the driving of the intelligent vehicle.

[0018] In one embodiment of the present invention, the vehicle motion prediction model is composed of an input layer, a fully connected layer, an output layer and a physical information layer; wherein the input layer, the fully connected layer and the output layer constitute a deep neural network; wherein the input layer is used to obtain the vehicle state quantity and the vehicle control quantity at the current time point and the historical time point, wherein the vehicle state quantity includes the vehicle motion state and the tire force state; and the vehicle control quantity includes the front wheel steering angle and acceleration. The fully connected layer is connected to the input layer, and the fully connected layer is configured as a multi-layer hidden layer, each node in each layer is not connected to each other, but is connected to all nodes in the previous layer, and the fully connected layer is used to receive the vehicle state quantity and the vehicle control quantity, and extract the features related to the vehicle state quantity and the features related to the vehicle control quantity after linear transformation and nonlinear transformation. The output layer is used to receive the features related to the vehicle state quantity and the features related to the vehicle control quantity, and predict the vehicle state quantity and the vehicle control quantity at the future time point. The physical information layer includes the ordinary differential equations of four-wheel vehicle dynamics based on the four-wheel vehicle model, including the longitudinal force-motion balance equation, the lateral force-motion balance equation and the torque-rotation balance equation.

[0019] In one embodiment of the present invention, the motion controller includes: a ground adhesion coefficient identification module, which is used to identify the adhesion coefficient between the vehicle and the ground at the current time point or the next time point; a state observation module, which is used to obtain the control and state data of the vehicle from the on-board sensors of the intelligent vehicle in real time, and generate the vehicle state quantity and vehicle control quantity at the current time point and the historical time point; a reference path module, which is used to obtain the reference path of the vehicle; a transverse and longitudinal coupling MPC module, which is connected to the ground adhesion coefficient identification module, the state observation module, and the reference path module, and includes a vehicle motion prediction model, a constraint condition submodule, and an optimal solution submodule, wherein: the constraint condition submodule is used to receive the adhesion coefficient to construct a constraint condition; the vehicle motion prediction model receives the vehicle state quantity and vehicle control quantity at the current time point and the historical time point, and predicts the vehicle state quantity and vehicle control quantity at the future time point; the optimal solution submodule constructs a cost function based on the reference path and the predicted vehicle state quantity and vehicle control quantity at the future time point, and solves the cost function.

[0020] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned intelligent vehicle motion control method when executed by a processor.

[0021] Another aspect of the present invention provides a computer program product, which implements the steps of the above-mentioned intelligent vehicle motion control method when executed by a processor.

[0022] It can be seen from the above scheme that the advantages of the present invention are:

[0023] The intelligent vehicle motion control method disclosed by the present invention is based on a physical information neural network to construct a vehicle motion prediction model for predicting the vehicle state quantity and vehicle control quantity at a future time point. The first data set constructed in a virtual environment is input into the constructed vehicle motion prediction model for pre-training, and the vehicle motion prediction model is fine-tuned using the second data set obtained in a real environment. The vehicle motion prediction model is integrated into the vehicle motion controller to control the vehicle driving. The vehicle motion prediction model introduces the dynamic differential equation into the loss function on the basis of DNN, making the network more robust and achieving better prediction of the vehicle state in different environments with less data. The intelligent vehicle motion control method has stronger environmental adaptability, more accurate optimization control results, more reliable satisfaction of constraints, and greatly improves the tracking accuracy, driving stability and safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram showing the structure of an intelligent vehicle motion control device provided by an embodiment of the present invention is shown;

[0025] Figure 2 A schematic diagram showing the overall flow of an intelligent vehicle motion control method provided by another embodiment of the present invention;

[0026] FIG. 3( a ) shows the internal structure of the vehicle motion prediction model 210

[0027] FIG. 3( b ) shows a specific schematic diagram of the physical information layer 214

[0028] Figure 4 shows the internal structure of the motion controller;

[0029] Reference numerals:

[0030] 100: Intelligent vehicles;

[0031] 110: computing unit;

[0032] 120: perception unit;

[0033] 130: navigation and positioning unit;

[0034] 140: body;

[0035] 150: Steering and driving unit;

[0036] 1501: steering mechanism;

[0037] 1502: driving brake mechanism;

[0038] 160: front wheel force sensor;

[0039] 170: rear wheel force sensor;

[0040] 180: chassis;

[0041] 200: horizontal and vertical coupling MPC module;

[0042] 210: Vehicle motion prediction model;

[0043] 211: input layer;

[0044] 212: fully connected layer;

[0045] 213: output layer;

[0046] 213A: force layer;

[0047] 213B: inertial layer;

[0048] 213C: sports layer;

[0049] 214: physical information layer;

[0050] 220: constraint condition submodule;

[0051] 230: optimal solution submodule;

[0052] 300: ground adhesion coefficient identification module;

[0053] 400: state observation module;

[0054] 500: yaw moment stability control module;

[0055] 600: Reference path module. DETAILED DESCRIPTION

[0056] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0057] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0058] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0059] Please refer to Figure 1 , which shows a schematic structural diagram of a motion control device for an intelligent vehicle 100 provided in one embodiment of the present invention.

[0060] Intelligent vehicles integrate multiple functions such as environmental perception, planning and decision-making, and multi-level assisted driving. They use a variety of technologies such as computers, modern sensors, information fusion, communications, artificial intelligence, and automatic control.

[0061] See also Figure 1 As shown in , the smart vehicle 100 includes a body 140, a chassis 180, a front wheel force sensor 160 installed on the front wheel, and a rear wheel force sensor 170 installed on the rear wheel. The motion control device includes: a computing unit 110, a sensing unit 120, a navigation and positioning unit 130, and a steering and driving unit 150. Among them, the computing unit 110 and the navigation and positioning unit 130 are fixed to the body 140 or the chassis 180, and the steering and driving unit 150 is fixed to the chassis 180.

[0062] Specifically, the navigation and positioning unit 130 is used to collect the current state of the intelligent vehicle and transmit it to the computing unit 110. The current state includes the vehicle's own posture and current vehicle speed. In practice, the navigation and positioning unit 130 is usually composed of an inertial element, a satellite positioning element, and a conditioning circuit.

[0063] The sensing unit 120 is used to sense the surrounding environment of the vehicle, determine the passable area in front and behind the vehicle, and send it to the calculation unit 110 to assist in determining the driving reference path. In practice, the sensing unit 120 is usually composed of at least one sensor of a camera, a laser radar, or a millimeter-wave radar installed on the outside of the vehicle body, and a sensor data conditioning circuit.

[0064] The computing unit 110 is electrically connected to the navigation and positioning unit 130, the perception unit 120, and the steering and driving unit 150. In practice, the computing unit is usually composed of a processor, a storage element, and a circuit board, and the processor and the storage element communicate through a high-speed bus. The processor is used for calculation, and the storage element is used to store programs and data. There are multiple I\O interfaces on the circuit board for communicating with other devices. In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). The storage element is used to store the software program that executes the reversing control strategy of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the following method embodiment, which is not specifically introduced here.

[0065] Specifically, in this embodiment, the computing unit 110 is the core unit of the motion control device, and its main configuration is: constructing a first data set using the control and state data of the virtual vehicle in the simulation environment; constructing a second data set using the control and state data of the intelligent vehicle in the real environment; and inputting the first data set into the constructed vehicle motion prediction model for pre-training, and using the second data set to fine-tune the vehicle motion prediction model to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer, and is used to predict the vehicle state quantity and vehicle control quantity at a future time point. The specific technical scheme of the computing unit will be described in detail in the subsequent motion control method, and only a brief introduction is given here.

[0066] The steering and drive unit 150 is equipped with a motion controller, which is integrated with a trained vehicle motion prediction model. The steering and drive unit is used to combine the real-time acquisition of control and state data of the intelligent vehicle, and use the motion controller to control the driving of the intelligent vehicle. In practice, the steering and drive unit 150 usually includes: a steering mechanism 1501 and a driving and braking mechanism 1502. The steering mechanism is used to drive the front wheels of the vehicle to achieve steering based on the steering control instructions generated by the motion controller, and the driving and braking mechanism is used to drive the vehicle wheels forward or backward, or brake the vehicle based on the driving control instructions generated by the motion controller, so as to achieve precise driving of the vehicle.

[0067] In addition, in specific matters, the communication method between the computing unit 110 and the steering and drive unit 150 may be CAN communication, Ethernet, direct physical connection, and other communication methods. For example, for a vehicle control-by-wire chassis equipped with a CAN bus, the computing unit 110 communicates with the steering and drive unit via the CAN bus. For a vehicle chassis that follows the Ethernet communication protocol, the computing unit 110 and the steering and drive unit communicate interactively via Ethernet; for a non-control-by-wire chassis, a direct physical connection between the computing unit 110 and the steering and drive unit 150 needs to be constructed.

[0068] Using the above-mentioned intelligent vehicle and motion control device, another embodiment of the present invention also discloses an intelligent vehicle motion control method based on physical information neural network, referring to Figure 2 As shown in Figure 2 The overall flow chart of the motion control method is shown.

[0069] An intelligent vehicle motion control method based on physical information neural network specifically comprises the following steps:

[0070] Step S1, collecting control and status data of a virtual vehicle in a simulation environment to construct a first data set; and collecting control and status data of an intelligent vehicle in a real environment to construct a second data set.

[0071] In this embodiment, for the control and state data of the virtual vehicle in the simulation environment, a virtual driving environment is created using intelligent driving simulation software, such as Simulink, Carla, Carsim, Gazebo, etc. Set different simulation conditions such as weather, road conditions, and traffic conditions. Select and configure a virtual vehicle model, including vehicle appearance, performance parameters, etc. Equip the vehicle model with corresponding virtual sensors, such as cameras, lidars, radars, etc., to obtain real-time status information of the vehicle. Run the virtual vehicle in the simulation environment to simulate various driving scenarios. Record the control input (such as acceleration, steering, braking, etc.) and state output (such as speed, position, posture, sensor data, etc.) of the vehicle. Then, the acquired control and state data of the virtual vehicle are cleaned and annotated, such as removing outliers and noise, ensuring the accuracy and consistency of the data, and formatting the data. Finally, the cleaned and annotated data is organized into a data set format to construct a first data set.

[0072] For the control and status data of smart vehicles in real environments, first deploy smart vehicles in real environments and ensure that various necessary sensors and actuators are installed on the vehicles. During the driving process of the vehicle, the control input and status output data of the vehicle are continuously recorded. Specifically, the CAN bus data can be read by connecting the computing unit to the vehicle CAN bus. The internal sensor information of the vehicle will be transmitted in the form of CAN messages. In addition to the common wheel speed, engine, brake and other information, the vehicle is usually equipped with an IMU, which can obtain the posture, speed and acceleration of the vehicle body on three axes. In addition, the information of the additional unit and the additional sensor can also be read. Ordinary vehicles can only obtain wheel speed information, but not wheel force information. Therefore, wheel force sensors can be installed, such as Figure 1 The front wheel force sensor 160 and the rear wheel force sensor 170 are then decoupled to obtain the force and torque information in three directions. In addition, the information of the perception unit 120 and the navigation positioning unit 130 can also be read. Then, the acquired control and status data of the intelligent vehicle are cleaned and annotated, such as removing outliers and noise, ensuring the accuracy and consistency of the data, and formatting the data. Finally, the cleaned and annotated data are organized into a data set format to construct a second data set.

[0073] Step S2: input the first data set into the constructed vehicle motion prediction model for pre-training, and use the second data set to fine-tune the vehicle motion prediction model to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer to predict the vehicle state quantity and vehicle control quantity at a future point in time.

[0074] In one embodiment, a vehicle motion prediction model is constructed based on a physics-informed neural network (PINN), so that the neural network model can learn changes in the global dynamic state of the vehicle, such as the nonlinear effects of tires, nonlinearity caused by suspension, load transfer, etc., and the robustness of the network can be guaranteed even when the amount of data is small. Specifically, as shown in reference FIG3(a), FIG3(a) shows the internal structure of a vehicle motion prediction model 210. The vehicle motion prediction model is composed of an input layer 211, a fully connected layer 212, an output layer 213, and a physical information layer 214. The input layer 211, the fully connected layer 212, and the output layer 213 constitute a deep neural network (DNN).

[0075] The input layer 211 is used to obtain the vehicle state and vehicle control quantity at the current time point and the historical time points. For the current time point t, its historical time points are the first, second, and last τ time points before t. Preferably, τ is 4. In order to achieve more accurate modeling of the nonlinear effects of multiple suspensions and multiple tires, the vehicle state quantity includes not only the vehicle motion state Also includes tire force status (F xfl ,F yfl ,F xfr ,F yfr ,F xrl ,F yrl ,F xrr ,F yrr ), where F xfl is the longitudinal force of the left front wheel, F yfl is the lateral force of the left front wheel, F xfr is the longitudinal force of the right front wheel, F yfr is the lateral force of the right front wheel, F xrl is the longitudinal force of the left rear wheel, F yrl is the lateral force of the left rear wheel, F xrr is the longitudinal force of the right rear wheel, F yrr Right rear wheel lateral force, v x is the longitudinal velocity of the center of mass, v y is the lateral velocity of the center of mass, r is the angular velocity of the center of mass, is the longitudinal acceleration of the center of mass, is the lateral acceleration of the center of mass, is the angular acceleration of the center of mass. The vehicle control quantity includes the front wheel steering angle and acceleration. The front wheel steering angle δ is the lateral control quantity, and the acceleration is the longitudinal control quantity. The advantage of using acceleration a to characterize the longitudinal control quantity is that the difference between the underlying power systems of electric vehicles and internal combustion engine vehicles does not need to be considered. The mapping relationship between acceleration and throttle and brake opening can be established using a calibration table or underlying control algorithm.

[0076] The fully connected layer 212 is connected to the input layer 211, and is used to receive the vehicle state quantity and the vehicle control quantity. After linear transformation and nonlinear transformation, the features related to the vehicle state quantity and the features related to the vehicle control quantity are extracted. Specifically, the fully connected layer 212 is a hidden layer of the DNN, which is characterized in that each node has no connection within the layer, but is connected to all nodes in the previous layer. Its forward calculation process is a linear weighted summation process. Each output can be regarded as each node of the previous layer multiplied by a weight coefficient, plus a bias value. Preferably, the number of hidden layers is 4.

[0077] The output layer 213 is connected to the fully connected layer 212, and is used to receive the features related to the vehicle state and the features related to the vehicle control, and predict the vehicle state and the vehicle control at a future time point. Specifically, the output layer includes a force layer 213A, an inertia layer 213B, and a motion layer 213C. The force layer 213A is used to predict the tire force state (F xfl ,F yfl ,F xfr ,F yfr ,F xrl ,F yrl ,F xrr ,F yrr ); The motion layer 213C is used to predict the motion state of the center of mass at a future point in time. The inertia layer 213B is used to predict the moment of inertia I at a future time point. z In this embodiment, the moment of inertia I z Use the predicted value of DNN. Because in addition to mass, mass distribution also affects the moment of inertia, making the moment of inertia of a vehicle difficult to measure accurately, and it changes with the load mass and the load mass distribution. Using traditional methods to calculate the moment of inertia of a vehicle is a cumbersome task, requiring disassembly of the vehicle, accurate weighing and positioning of each component, and the use of CAD software or multi-body physics simulation. In the present invention, a more accurate moment of inertia I can be obtained. z , when the load mass and load mass distribution change, the predicted moment of inertia I z Can also change accordingly.

[0078] Referring to FIG. 3( b ), FIG. 3( b ) shows a specific schematic diagram of the physical information layer 214. The physical information layer 214 is connected to the force layer 213A, the inertia layer 213B and the motion layer 213C of the output layer 213. The physical information layer 214 is configured with a four-wheel vehicle dynamics ordinary differential equation based on a four-wheel vehicle model, including a longitudinal force-motion balance equation, a lateral force-motion balance equation and a torque-rotation balance equation, namely:

[0079]

[0080] The three equations from top to bottom are the longitudinal force-motion balance equation, the lateral force-motion balance equation, and the torque-rotation balance equation. xfl is the longitudinal force of the left front wheel, F yfl is the lateral force of the left front wheel, F xfr is the longitudinal force of the right front wheel, F yfr is the lateral force of the right front wheel, F xrl is the longitudinal force of the left rear wheel, F yrl is the lateral force of the left rear wheel, F xrris the longitudinal force of the right rear wheel, F yrr Right rear wheel lateral force, v x is the longitudinal velocity of the center of mass, v y is the lateral velocity of the center of mass, r is the angular velocity of the center of mass, is the longitudinal acceleration of the center of mass, is the lateral acceleration of the center of mass, is the angular acceleration of the center of mass, δ is the front wheel turning angle, m is the vehicle mass, l s is half wheelbase, l f is the distance from the center of mass to the front axle, l r is the distance from the center of mass to the rear axle, I z In addition, in Figure 3(b), β is the center of mass slip angle, φ is the heading angle, and α is the tire slip angle.

[0081] The vehicle motion state at the future time point output by the output layer 213 Tire force state at a future point in time (F xfl ,F yfl ,F xfr ,F yfr ,F xrl ,F yrl ,F xrr ,F yrr ), the moment of inertia I at the future point z The input is into the four-wheel vehicle dynamics ordinary differential equation based on the four-wheel vehicle model configured in the physical information layer 214, and then the loss function of the physical information layer can be obtained, which is further fed back to the fully connected layer 212 for parameter tuning.

[0082] In this embodiment, the total loss function of the vehicle motion prediction model includes two parts: the first loss function of the deep neural network (DNN) and the second loss function of the physical information layer. data , which is determined based on the root mean square error between the vehicle state and the vehicle control at the predicted future time point. For the second loss function L of the physical information layer phys , input the predicted vehicle state and vehicle control quantity at the future time point, as well as the vehicle steering angle and vehicle mass into the three equations of the longitudinal force-motion balance equation, the lateral force-motion balance equation and the torque-rotation balance equation of the four-wheel vehicle dynamics ordinary differential equation, and obtain three residuals, and then take the root mean square error of the three residuals to obtain the second loss function of the physical information layer. Finally, the total loss function Loss of the vehicle motion prediction model is determined based on the weighted sum of the first loss function and the second loss function. Loss=λ data L data +λ phys L phys , where λ data ,phys L data , L phys The weight coefficient of .

[0083] In one embodiment, there are two ways to obtain the vehicle mass m. One is to weigh the vehicle after it is loaded and record the total mass of the vehicle at that time. However, this method relies on a scale and cannot make an accurate estimate if the loaded mass changes after the vehicle sets off. Another method is to obtain vertical force data through the information of the force sensors installed on the four wheels, combined with the onboard IMU information, and convert it into the mass m of the vehicle. The vehicle mass obtained by this method can be updated in real time according to the change of the loaded mass.

[0084] In addition, the training of the vehicle motion prediction model 210 adopts the backpropagation algorithm (Backpropagation) in combination with gradient descent (Gradient Descent) or other optimization algorithms (such as Adam, Adagrad, etc.). By calculating the gradient of the loss function relative to the weight, the network weights are updated to minimize the total loss function, thereby achieving a good fit to the training data.

[0085] In addition, when the vehicle motion prediction model 210 is used for reasoning, only a deep neural network (DNN) is needed, and the physical information layer 214 is no longer needed. The vehicle motion prediction model 210 self-loop reasoning is to predict the vehicle state and vehicle control at the future time point based on the vehicle state and vehicle control at the current time point and the historical time point. The specific process is as follows: at time point t, the vehicle state and vehicle control observed or recorded from t-τ to t are used as input, and the output is obtained through the reasoning of DNN, which is the vehicle state at the next time point of t (i.e., t+1 time point). Then the vehicle state at t+1 time point and the vehicle control at that time point (specifically obtained by the optimal solution submodule 230), plus the vehicle state and vehicle control at t-τ+1 to t time point, are used as the input of the deep neural network (DNN), and the vehicle state at the next time point (i.e., t+2 time point) is predicted, and this cycle is repeated until the required vehicle state in the future time domain H is obtained.

[0086] In addition, since the output of DNN does not contain position information, it is necessary to use the initial position and predicted speed to obtain the position of each time point in the predicted time domain. Since the output of the neural network is in the vehicle coordinate system, and the vehicle position is expressed in the world coordinate system, it is necessary to perform the corresponding coordinate system conversion through existing technical means.

[0087] In this embodiment, the first data set constructed based on the virtual environment is further input into the vehicle motion prediction model constructed above for pre-training. Then, the vehicle motion prediction model is fine-tuned using the second data set constructed based on the real environment to obtain a trained vehicle motion prediction model.

[0088] Step S3: Integrate the trained vehicle motion prediction model into the motion controller of the intelligent vehicle, and use the motion controller to control the driving of the intelligent vehicle.

[0089] In this embodiment, the trained vehicle motion prediction model is further integrated into the motion controller of the intelligent vehicle. Figure 4 As shown in Figure 4 The internal structure of the motion controller is shown. The motion controller includes: a transverse and longitudinal coupling MPC module 200, a ground adhesion coefficient identification module 300, a state observation module 400, a reference path module 600, etc. The transverse and longitudinal coupling MPC module 200 specifically includes a vehicle motion prediction model 210, a constraint condition submodule 220, and an optimal solution submodule 230.

[0090] The ground adhesion coefficient identification module 300 is used to identify the adhesion coefficient μ between the vehicle and the ground at the current time point or the next time point, and transmit it to the constraint condition 220 submodule of the lateral and longitudinal coupling MPC module 200. In one embodiment, based on the vehicle control and state data x(t) obtained from the on-board sensor of the intelligent vehicle 100, combined with the tire model, the current ground adhesion coefficient μ is estimated by an extended Kalman filter method. Preferably, since intelligent vehicles are usually equipped with high-resolution visual sensors, a vision-based ground adhesion coefficient identifier can also be used to predict the future adhesion coefficient μ.

[0091] The state observation module 400 is used to obtain the vehicle control and state data x(t) from the on-board sensors of the intelligent vehicle in real time, and generate the vehicle state quantity and vehicle control quantity at the current time point and the historical time point. And transmitted to the vehicle motion prediction model 210 of the lateral and longitudinal coupling MPC module 200.

[0092] The reference path module 600 provides the optimal solution submodule 230 of the lateral and longitudinal coupling MPC module 200 with a reference path for the vehicle to move forward or backward. The reference path in the world coordinate system is usually Contains a sequence of n reference points: The reference path can be obtained by one or more of the following methods, for example: extracting a section of the stored global path as a reference path; when the vehicle is driving, storing the path passed in real time and taking a section of it as a reference path; using a perception unit to perceive the passable area, and combining the global path to plan a reference path in real time.

[0093] The constraint condition submodule 220 of the lateral and longitudinal coupling MPC module 200 is used to receive the adhesion coefficient to construct constraint conditions; the vehicle motion prediction model 210 receives the vehicle state quantity and vehicle control quantity at the current time point and the historical time point, and predicts the vehicle state quantity and vehicle control quantity at the future time point; the optimal solution submodule 230 constructs a cost function based on the reference path and the predicted vehicle state quantity and vehicle control quantity at the future time point, and solves the cost function at the ground adhesion coefficient μ.

[0094] The cost function consists of two parts: tracking cost and driving cost. The tracking cost is constructed based on the error between the predicted vehicle position at a future time point and the corresponding waypoint on the reference path. The driving cost is constructed based on the change in acceleration and the change in the front wheel steering angle within a given time period. The final constructor is:

[0095]

[0096] In the formula, x h represents the predicted x coordinate of the vehicle at the hth time point, x ref,h is the x coordinate of the reference waypoint corresponding to the time point, y h Represents the predicted y coordinate of the vehicle at that time, y ref,h is the y coordinate of the reference waypoint corresponding to that time point, Δa h Indicates the change in vehicle acceleration from h to h+1, Δδ h Indicates the change in the vehicle steering angle during this period of time. h ,y h The vehicle position is predicted by the vehicle motion prediction model 210, which is determined based on the vehicle state. The first item in the constructor is the tracking cost item, using the cost matrix Penalize the error between the vehicle position and the corresponding waypoint on the reference path. The latter term is the driving cost term, using the cost matrix Penalizes rapid changes in acceleration and steering. The cost function includes both lateral and longitudinal control quantities, thus achieving lateral and longitudinal coupled control.

[0097] By solving the optimal vehicle control quantity u0(t) (including the front wheel steering angle and acceleration, etc.), the minimum value of the following cost function is obtained in the entire forward prediction time domain H. In the optimal solution process, it is necessary to add constraints to prevent roll and sideslip, which are provided by the constraint condition 220 submodule. The ground adhesion coefficient μ required for anti-sideslip is provided by the ground adhesion coefficient identification 300 module. If the ground adhesion coefficient identification 300 module can only provide the ground adhesion coefficient μ at the current time point, then the ground adhesion coefficient μ at the current time point is used for the calculation of the prediction time domain constraint conditions, which will produce a certain error; if the ground adhesion coefficient identification 300 module can predict the ground adhesion coefficient μ at a future time point, then using it for the prediction time domain can reduce the error. Therefore, in this embodiment, the ground adhesion coefficient μ provided by the ground adhesion coefficient identification 300 module is only used to construct constraint conditions, and is not used for the prediction model, so it will not affect the prediction accuracy of the prediction model.

[0098] In addition, after the optimal solution submodule 230 obtains the optimal vehicle control amount u0(t), the optimal vehicle control amount u0(t) is further fed back to the input layer of the vehicle motion prediction model 210 to execute the DNN reasoning process.

[0099] In addition, further reference Figure 4 As shown in , in some embodiments, the motion controller further includes a yaw moment stabilization control module 500, which is connected to the output layer of the vehicle motion prediction model 210, receives the predicted vehicle state quantity and vehicle control quantity at the future time point, performs predictive yaw moment control, and obtains the yaw moment control quantity u′, and the yaw moment control quantity u′ is superimposed with the optimal vehicle control quantity u0(t) obtained by the optimal solution submodule 230 to obtain the optimized control quantity u * (t) The value is further fed back to the intelligent vehicle to control the driving of the intelligent vehicle. In this embodiment, the vehicle stability and safety are improved by adding the yaw moment control amount u'.

[0100] Ultimately, the intelligent vehicle combines the control and status data acquired in real time, and tracks the reference path under the joint action of the predicted vehicle status at the future point in time, the optimized vehicle control quantity, the reference path, etc., to obtain the maximum allowable vehicle speed while ensuring safety.

[0101] In summary, the motion control method of the intelligent vehicle disclosed in the present invention combines a deep neural network and a physical information layer to construct a vehicle motion prediction model for predicting the vehicle state and vehicle control at future points in time. Based on DNN, the dynamic differential equation is introduced into the loss function to make the network more robust, and better prediction of the vehicle state in different environments can be achieved with less data. The motion control method of the intelligent vehicle has stronger environmental adaptability, more accurate optimization control results, and more reliable satisfaction of the constraints, which greatly improves the tracking accuracy, driving stability and safety of the vehicle.

[0102] In addition, the present invention also provides a computer program product in one embodiment, which includes a computer program. The computer program can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the motion control methods provided by the above methods.

[0103] The present invention also proposes a storage medium in another embodiment for storing a computer program for executing the motion control method. It should be understood that the storage medium in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0104] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. An intelligent vehicle motion control method based on physical information neural network, characterized in that: Include: Collect control and state data of the virtual vehicle in a simulation environment to construct a first data set; Collect control and status data of intelligent vehicles in real environments and construct a second data set; The first data set is input into a constructed vehicle motion prediction model for pre-training, and the vehicle motion prediction model is fine-tuned using the second data set to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer for predicting a vehicle state quantity and a vehicle control quantity at a future time point; The trained vehicle motion prediction model is integrated into the motion controller of the intelligent vehicle, and the motion controller is used to control the driving of the intelligent vehicle.

2. The method according to claim 1, characterized in that The vehicle motion prediction model is composed of an input layer, a fully connected layer, an output layer and a physical information layer; wherein the input layer, the fully connected layer and the output layer constitute a deep neural network; The vehicle state quantity and vehicle control quantity at the current time point and the historical time point are obtained in the input layer, wherein the vehicle state quantity includes the vehicle motion state and the tire force state; the vehicle control quantity includes the front wheel steering angle and acceleration; The fully connected layer receives the vehicle state quantity and the vehicle control quantity, and extracts the features related to the vehicle state quantity and the features related to the vehicle control quantity after linear transformation and nonlinear transformation; The output layer receives the features related to the vehicle state and the features related to the vehicle control, and predicts the vehicle state and the vehicle control at a future time point; The physical information layer is configured with a four-wheel vehicle dynamics ordinary differential equation based on a four-wheel vehicle model, including a longitudinal force-motion balance equation, a lateral force-motion balance equation and a torque-rotation balance equation.

3. The method according to claim 2, characterized in that The output layer includes a force layer, an inertia layer and a motion layer; wherein, the tire force state at a future time point is predicted in the force layer; Predict the moment of inertia at future points in the inertial layer; The motion layer predicts the vehicle motion state at a future time point with respect to the center of mass motion state.

4. The method according to claim 2 or 3, characterized in that: Determining a first loss function of the deep neural network according to a root mean square error between a vehicle state quantity and a vehicle control quantity at a predicted future time point; Input the predicted vehicle state quantity and vehicle control quantity at the future time point, as well as the vehicle steering angle and vehicle mass into the four-wheel vehicle dynamics ordinary differential equation to obtain three residuals; take the root mean square error of the three residuals to obtain the second loss function of the physical information layer; Determine a total loss function of the vehicle motion prediction model based on a weighted sum of the first loss function and the second loss function.

5. The method according to claim 2 or 3, characterized in that: The motion controller comprises a ground adhesion coefficient identification module, a state observation module, a reference path module, and a transverse and longitudinal coupling MPC module, wherein the transverse and longitudinal coupling MPC module comprises a vehicle motion prediction model, a constraint condition submodule, and an optimal solution submodule; identifying the adhesion coefficient between the vehicle and the ground at a current time point or a next time point in the ground adhesion coefficient identification module; The constraint condition submodule receives the adhesion coefficient to construct a constraint condition; The state observation module generates the vehicle state quantity and vehicle control quantity at the current time point and the historical time point according to the vehicle control and state data obtained from the on-board sensors of the intelligent vehicle in real time; The vehicle motion prediction model receives the vehicle state quantity and vehicle control quantity at the current time point and the historical time point, and predicts the vehicle state quantity and vehicle control quantity at the future time point; Generate a reference path for the vehicle in a reference path module; In the optimal solution submodule, a cost function is constructed according to the reference path and the vehicle state quantity and the vehicle control quantity at the future time point, and the cost function is solved based on the constraint conditions.

6. The method according to claim 5, characterized in that Constructing a tracking cost term according to an error between a predicted vehicle position at a future time point and a corresponding waypoint of the reference path; the vehicle motion state includes the vehicle position; Construct a driving cost term according to the change in acceleration and the change in the front wheel steering angle within a given time period; The cost function is obtained according to the tracking cost term and the driving cost term.

7. The method according to claim 5, characterized in that The motion controller also includes a yaw moment stabilization control module; The yaw moment stabilization control module receives the predicted vehicle state quantity and vehicle control quantity at a future time point and performs predictive yaw moment control.

8. An intelligent vehicle motion control device based on physical information neural network, characterized in that: Include: A computing unit, configured to construct a first data set using control and state data of a virtual vehicle in a simulation environment; and to construct a second data set using control and state data of an intelligent vehicle in a real environment; as well as, The first data set is input into a constructed vehicle motion prediction model for pre-training, and the vehicle motion prediction model is fine-tuned using the second data set to obtain a trained vehicle motion prediction model; wherein the vehicle motion prediction model integrates a deep neural network and a physical information layer for predicting a vehicle state quantity and a vehicle control quantity at a future time point; The steering and driving unit is equipped with a motion controller which integrates a trained vehicle motion prediction model; the steering and driving unit is used to acquire control and status data of the intelligent vehicle in real time and use the motion controller to control the driving of the intelligent vehicle.

9. The device according to claim 8, characterized in that The vehicle motion prediction model is composed of an input layer, a fully connected layer, an output layer and a physical information layer; wherein the input layer, the fully connected layer and the output layer constitute a deep neural network; The input layer is used to obtain the vehicle state and vehicle control at the current time point and the historical time point, wherein the vehicle state includes the vehicle motion state and tire force state; the vehicle control includes the front wheel steering angle and acceleration; The fully connected layer is used to receive the vehicle state quantity and the vehicle control quantity, and extract the features related to the vehicle state quantity and the features related to the vehicle control quantity after linear transformation and nonlinear transformation; The output layer is used to receive the features related to the vehicle state quantity and the features related to the vehicle control quantity, and predict the vehicle state quantity and the vehicle control quantity at a future time point; The physical information layer is configured with four-wheel vehicle dynamics ordinary differential equations based on a four-wheel vehicle model, including a longitudinal force-motion balance equation, a lateral force-motion balance equation and a torque-rotation balance equation.

10. The device according to claim 9, characterized in that The motion controller comprises: A ground adhesion coefficient identification module is used to identify the adhesion coefficient between the vehicle and the ground at the current time point or the next time point; The state observation module is used to obtain the control and state data of the vehicle from the on-board sensors of the intelligent vehicle in real time, and generate the vehicle state quantity and vehicle control quantity at the current time point and the historical time point; A reference path module is used to obtain a reference path for the vehicle to travel; The transverse and longitudinal coupling MPC module is connected to the ground adhesion coefficient identification module, the state observation module, and the reference path module, and includes a vehicle motion prediction model, a constraint condition submodule, and an optimal solution submodule, wherein: The constraint condition submodule is used to receive the adhesion coefficient to construct the constraint condition; The vehicle motion prediction model receives the vehicle state quantity and vehicle control quantity at the current time point and the historical time point, and predicts the vehicle state quantity and vehicle control quantity at the future time point; The optimal solution submodule constructs a cost function based on the reference path and the predicted vehicle state quantity and vehicle control quantity at a future time point, and solves the cost function.

11. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

12. A computer program product, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images