Differential game approach to solving the shared control problem of autonomous vehicles

Abstract

This application proposes a differential game method to solve the problem of shared control between humans and machines in autonomous driving vehicles, including: establishing a non-cooperative differential game model between humans and machines based on the driver's target trajectory and the trajectory planned by the autonomous driving system, and considering the control constraints in the form of matrix inequalities, wherein the non-cooperative differential game model uses Nash equilibrium to represent the human-machine game problem; introducing function approximation methods such as neural networks based on the non-cooperative differential game model to establish a differential game system with unknown parameters; using an integral parallel learning method, using trajectory data as numerical product to identify unknown parameters in the system; based on the identified differential game system, learning the Nash equilibrium solution in the human-machine game problem through an adaptive iterative learning algorithm. This application using the above scheme improves the control strategy of the driving system by solving the non-cooperative Nash equilibrium solution of humans and machines, without relying on the precise parameters of the system model.

Description

Technical Field

[0001] The present application relates to the fields of control theory and artificial intelligence technology, and in particular to a differential game method and device for solving the problem of human-machine shared control of autonomous driving vehicles. Background Art

[0002] In the context of the current development of intelligent transportation, human-machine shared control has gradually become one of the important ways to achieve intelligent vehicle driving. In human-machine shared control, the driver and the automatic driving system cooperate to control the vehicle, which combines the flexibility of manual driving and the precise control of the automatic system. How to solve the effective coordination between the driver and the automatic driving system has become a key issue in human-machine shared control. Existing methods include rule-based and game-based methods: the rule-based method has a fixed strategy and is difficult to adapt to complex dynamic environments; the differential game-based method uses differential equations to describe the dynamic system of the vehicle, regards the driver and the automatic system as game players, and calculates the Nash equilibrium strategy to enable both humans and machines to reach a relatively optimal control state, thereby effectively solving the conflict and coordination problems in the decision-making process.

[0003] Existing methods such as model predictive control mostly rely on accurate vehicle dynamics models. However, in practical applications, accurate models are often difficult to obtain, which limits the performance of model-based methods in complex and dynamic environments. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first purpose of this application is to propose a differential game method to solve the problem of shared control between humans and machines in autonomous driving vehicles. It solves technical problems that are difficult to achieve with existing methods. It can consider the situation where control is constrained without relying on the precise parameters of the system model, and improve the control strategy of the driving system by solving the non-cooperative Nash equilibrium solution of humans and machines.

[0006] The second purpose of this application is to propose a differential game device to solve the problem of human-machine shared control of autonomous driving vehicles.

[0007] The third objective of the present application is to provide a computer device.

[0008] A fourth object of the present application is to provide a non-transitory computer-readable storage medium.

[0009] A fifth object of the present application is to provide a computer program product.

[0010] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present application proposes a differential game method for solving the problem of shared control between humans and machines in autonomous driving vehicles, including: establishing a non-cooperative differential game model between humans and machines based on the driver's target trajectory and the trajectory planned by the autonomous driving system, and considering the control constraints in the form of matrix inequalities, wherein the non-cooperative differential game model uses Nash equilibrium to represent the human-machine game problem; based on the non-cooperative differential game model, function approximation methods such as neural networks are introduced to establish a differential game system with unknown parameters; an integral parallel learning method is adopted to use trajectory data as numerical products to identify unknown parameters in the system; based on the identified differential game system, the Nash equilibrium solution in the human-machine game problem is learned through an adaptive iterative learning algorithm.

[0011] Optionally, in one embodiment of the present application, a non-cooperative differential game model is established, including:

[0012] Build a vehicle control system model;

[0013] Based on the vehicle control system model, the control inputs of the driver and the automatic driving system are integrated as the shared control inputs of the human-machine shared control system, and a human-machine shared control differential game system is established;

[0014] Set the permissible control set and cost function for the driver and the autonomous driving system;

[0015] Assuming that the goals of the driver and the autonomous driving system are to minimize their respective cost functions based on the given differential game system and matrix inequality constraints, and that the driver and the autonomous driving system will not cooperate or negotiate, the game problem is transformed into solving the Nash equilibrium solution based on the assumption, and the necessary conditions satisfied by the game problem are determined.

[0016] Optionally, in one embodiment of the present application, the vehicle control system model is:

[0017]

[0018] in, is the system state variable, is the control input, is the system drift vector function, is the control vector function of the system, is the initial state of the system;

[0019] The shared control input is:

[0020]

[0021] in, , denote the control inputs of the driver and the autonomous driving system, respectively, Assigning coefficients to the authority imposed on the driver and the automated driving system;

[0022] The human-computer shared control differential game system is:

[0023]

[0024]

[0025] in, is the output variable of the system, is the output matrix of the system;

[0026] The admissible control sets are:

[0027]

[0028]

[0029] in, and are the constraint matrices for the driver and the autonomous driving system, respectively. and are the threshold vectors for the driver and the autonomous driving system, respectively;

[0030] The cost function is a quadratic cost function, expressed as:

[0031]

[0032]

[0033] in, is the driver’s target trajectory, The target trajectory planned by the autonomous driving system, and are the output tracking error weight matrices for the driver and the autonomous driving system, and are the control energy weight matrices for the driver and the autonomous driving system, For a given finite time domain;

[0034] The game problem is transformed into a solution to the Nash equilibrium solution, and the formula is:

[0035]

[0036]

[0037] The necessary conditions satisfied by the game problem include the first set of equations and the second set of equations, where the first set of equations is:

[0038]

[0039]

[0040]

[0041] The second set of equations is:

[0042]

[0043]

[0044]

[0045] in, and are the co-state quantities of the driver and the automatic driving system, and are the Hamiltonian functions of the driver and the autonomous driving system respectively, expressed as;

[0046]

[0047]

[0048] Optionally, in one embodiment of the present application, a differential game system with unknown parameters is established, including:

[0049] The unknown differential game system is expressed as:

[0050]

[0051]

[0052] in, and is the unknown weight matrix, and is the activation function, and is the corresponding approximation error;

[0053] make is the total number of neurons, is the augmented weight matrix, is the augmented activation function vector, is the lumped error, and the unknown differential game system is expressed as:

[0054]

[0055] When the true weight system matrix is ​​unknown, the approximate system is defined as:

[0056]

[0057] in, is the estimated value of the weight coefficient matrix.

[0058] Optionally, in one embodiment of the present application, identifying unknown parameters in the system includes:

[0059] Collect W group The historical trajectory data in is recorded as , and for each set of data, calculate:

[0060]

[0061] in, ;

[0062] Defining the Matrix , the collected historical trajectory data set meets , the integral parallel learning method is used to simultaneously use the numerical integration of the historical data set and the real-time data to update the unknown parameters in the differential game system. When updating, the parameter adaptation law is:

[0063]

[0064] in, , is the learning rate, is the estimated error of the system state during the i-th iteration.

[0065] Optionally, in one embodiment of the present application, learning a Nash equilibrium solution in a human-computer game problem includes:

[0066] Step S1: Set i=0; initialize coefficients , , , and convergence accuracy , , select the initial control input of the driver and the automatic control system and and the initial coefficient matrix estimates , solve the following set of equations to update the system's initial trajectory estimate and initial weight coefficient matrix estimate:

[0067]

[0068] in, and are the estimation errors of the initial system state trajectory and the initial weight coefficient matrix, The specific expression of is determined by the adaptive law in the integral parallel learning method, is the actual initial state trajectory of the system;

[0069] Step S2: Calculate by the following formula :

[0070]

[0071] in, represents the estimated value of the Hamiltonian function;

[0072] Step S3: Update the driver's control input by solving the following equation:

[0073]

[0074] in, , ;

[0075] Step S4: Calculate the new system state through the updated control input And the corresponding driver cost function , determine whether the updated driver control input reduces its cost function. If , then let , , is the iteration step, return to step S2, otherwise, go to step S5;

[0076] Step S5: Calculate by the following formula :

[0077]

[0078] in, represents the estimated value of the Hamiltonian function;

[0079] Step S6: Update the control input of the autonomous driving system by solving the following equation:

[0080]

[0081] in, , ;

[0082] Step S7: Calculate the new system state through the updated control input And the corresponding cost function of the autonomous driving system , to determine whether the updated control input of the autonomous driving system reduces its cost function. If , then let , , is the iteration step, return to step S6, otherwise, go to step S8;

[0083] Step S8: Update the system trajectory estimate and weight coefficient matrix estimate by solving the following equations:

[0084]

[0085] Step S9: If , then go to step S10, otherwise, let , , , , , and return to step S2;

[0086] Step S10: If , stop the iteration process and get the corresponding Nash equilibrium solution, otherwise, let , , , , , and return to step S2.

[0087] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a differential game device for solving the problem of human-machine shared control of an autonomous driving vehicle, comprising:

[0088] A system building module is used to establish a human-machine non-cooperative differential game model based on the driver's target trajectory and the trajectory planned by the autonomous driving system, taking into account the control constraints in the form of matrix inequalities. The non-cooperative differential game model uses Nash equilibrium to represent the human-machine game problem.

[0089] The system building module is also used to introduce function approximation methods such as neural networks based on the non-cooperative differential game model to establish a differential game system with unknown parameters;

[0090] The system identification module is used to identify unknown parameters in the system by using the integral parallel learning method and the trajectory data for numerical integral identification;

[0091] The computational solution module is used to learn the Nash equilibrium solution in the human-computer game problem through an adaptive iterative learning algorithm based on the identified differential game system.

[0092] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned differential game method for solving the problem of human-machine shared control of autonomous driving vehicles is implemented.

[0093] In order to achieve the above-mentioned objectives, the fourth aspect embodiment of the present application proposes a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor, the above-mentioned differential game method for solving the human-machine shared control problem of autonomous driving vehicles can be executed.

[0094] In order to achieve the above-mentioned objectives, the fifth aspect embodiment of the present application proposes a computer program product, which, when the instruction processor in the computer program product is executed, executes the above-mentioned differential game method for solving the human-machine shared control problem of autonomous driving vehicles.

[0095] The differential game method and device for solving the problem of shared control between man and machine in autonomous driving vehicles in the embodiment of the present application establishes a non-cooperative differential game model of the two based on the target trajectory of the driver and the trajectory planned by the autonomous driving system and the control constraints in the form of matrix inequalities; introduces function approximation methods such as neural networks based on the model of non-cooperative differential games to establish a parameter identification model of the unknown game system; uses an integral parallel learning method to perform numerical integration using real-time trajectory data and historical trajectory data to identify unknown parameters in the model; based on the identified model, proposes an adaptive iterative learning algorithm to learn the Nash equilibrium solution in the man-machine game problem. This application takes into account the situation where the control is constrained without relying on the precise parameters of the system model, and improves the control strategy of the driving system by solving the non-cooperative Nash equilibrium solution of man and machine.

[0096] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0098] Figure 1 A flowchart of a differential game method for solving the human-machine shared control problem of an autonomous driving vehicle provided in the first embodiment of the present application;

[0099] Figure 2 A schematic diagram of a vehicle yaw-sideslip "bicycle" model according to an embodiment of the present application;

[0100] Figure 3 A driving scene design diagram for an embodiment of the present application;

[0101] Figure 4 It is an iterative diagram of the change of the cost function of the driver and the automatic driving system under different weights in an embodiment of the present application;

[0102] Figure 5 A comparison diagram of the actual output trajectory of the vehicle under different weights in an embodiment of the present application;

[0103] Figure 6 A comparison diagram of control inputs of a human driver and an autonomous driving system under different weights in an embodiment of the present application;

[0104] Figure 7 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0105] Figure 8 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0106] Fig. 9 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0107] Fig.10 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0108] Fig.11 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0109] Fig.12 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0110] Fig.13 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0111] Fig.14 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0112] Fig.15 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0113] Fig.16 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0114] Fig.17 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0115] Fig.18For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0116] Fig.19 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0117] Fig. 20 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0118] Fig.21 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0119] Fig. 22 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0120] Fig.23 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0121] Fig.24 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0122] Fig.25 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0123] Fig.26 For the embodiments of this application Comparison diagram between the actual state trajectory and the estimated trajectory of the vehicle;

[0124] Fig. 27 A 3D schematic diagram of vehicle trajectory changes under different weights in an embodiment of the present application;

[0125] Fig.28 A schematic diagram of the structure of a differential game device for solving the problem of human-machine shared control of an autonomous driving vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0126] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0127] The following describes, with reference to the accompanying drawings, a differential game method and device for solving the problem of human-machine shared control of an autonomous driving vehicle according to an embodiment of the present application.

[0128] Figure 1 A flow chart of a differential game method for solving the human-machine shared control problem of an autonomous driving vehicle provided in Example 1 of the present application.

[0129] like Figure 1 As shown, the differential game method for solving the human-machine shared control problem of the autonomous driving vehicle includes the following steps:

[0130] Step 101, establishing a human-machine non-cooperative differential game model based on the driver's target trajectory and the trajectory planned by the automatic driving system and considering the control constraints in the form of matrix inequalities, wherein the non-cooperative differential game model uses Nash equilibrium to represent the human-machine game problem;

[0131] In this embodiment, the following vehicle control system model is considered:

[0132]

[0133] in, is the system state variable, It is a human-machine shared control input. is the system drift vector function, is the control vector function of the system, is the initial state of the system;

[0134] For human-machine shared control systems, the collaborative control of the driver and the autonomous driving system can be achieved by weighted fusion of the control inputs of the two, that is, the shared control input can be expressed as:

[0135]

[0136] in, and denote the control inputs of the driver and the autonomous driving system, respectively, is the authority allocation coefficient imposed on the driver and the autonomous driving system, It is fully manual driving. It is fully autonomous driving;

[0137] Therefore, the vehicle-machine shared control differential game system can be described as:

[0138]

[0139]

[0140] in, is the output variable of the system, is the output matrix of the system;

[0141] Assume that the permissible control set of the driver and the autonomous driving system can be expressed as:

[0142]

[0143]

[0144] in, and are the constraint matrices for the driver and the autonomous driving system, respectively. and are the threshold vectors for the driver and the autonomous driving system, respectively;

[0145] For the driver and the autonomous driving system, the following quadratic cost functions are used respectively:

[0146]

[0147]

[0148] in, is the driver’s target trajectory, The target trajectory planned by the autonomous driving system, and Output tracking error weight matrices for humans and autonomous driving systems respectively, and The energy weight matrices for human and autonomous driving system control are: For a given finite time domain, assuming that the goals of the driver and the autonomous driving system are to minimize their respective cost functions based on the given differential game system and matrix inequality constraints, and the driver and the autonomous driving system will not cooperate or negotiate, then the game problem can be solved by finding the Nash equilibrium solution, which is formulated as:

[0149]

[0150]

[0151] The Pontryagin minimum principle gives the necessary conditions for the above two-person non-zero-sum differential game problem to be satisfied as the following two sets of equations:

[0152]

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] in, and are the co-state quantities of the driver and the automatic driving system, and are the Hamiltonian functions of the driver and the autonomous driving system, respectively, and are defined as:

[0159]

[0160]

[0161] In this embodiment, a yaw-sideslip "bicycle" model system is taken as an example, and its schematic diagram is as follows: Figure 2 shown.

[0162] The vehicle dynamics equation described by the yaw-sideslip "bicycle" model is:

[0163]

[0164]

[0165] in, represents the lateral velocity in the vehicle coordinate system, represents the yaw rate, Indicates human-machine shared control input, is the longitudinal velocity in the vehicle coordinate system, and are the front wheel steering stiffness and the rear wheel steering stiffness, is the vehicle mass, and are the distances from the vehicle center of mass to the front and rear axles, respectively. is the moment of inertia, is the steering gear ratio;

[0166] In the global coordinate system, the displacement of the vehicle can be expressed as:

[0167]

[0168]

[0169] in, is the yaw angle of the vehicle. When the yaw angle is small, the displacement of the vehicle can be further approximated as:

[0170]

[0171]

[0172] The differential game system equation of human-machine shared control can be described as:

[0173]

[0174]

[0175] in, is the state variable of the system, due to the longitudinal displacement of the vehicle and yaw angle It is easy to measure, so it is generally selected is the output variable of the system,

[0176]

[0177]

[0178]

[0179] They are the state matrix, input matrix and output matrix of the game system respectively;

[0180] Taking the above as an example, the model in CarSim is selected as the simulation model, the vehicle model is selected as a Class C hatchback, the tire model is selected as 215 / 55 R17, and some parameters of the vehicle model are shown in the following table, Table 1:

[0181] Table 1

[0182]

[0183] Taking the above as an example, the driving scenario design is: Assume that the vehicle is driving Figure 3 On the two-lane road shown in the figure, there is an obstacle or other potential danger in front of the vehicle, and the driver fails to notice this situation. Therefore, the driver's intention is to maintain normal straight driving, such as Figure 3 As shown in Figure 2, the autonomous driving system detected this dangerous situation and planned a lane change path, as shown in Figure 2. Figure 3 As shown, the autonomous driving planning path can be expressed as:

[0184]

[0185] Taking the above example, the control inputs of the driver and the automatic driving system are selected to meet , , then the matrix inequality admissible control set can be expressed as:

[0186]

[0187]

[0188] Taking the above as an example, the cost function of the driver and the automatic driving system is selected as:

[0189]

[0190]

[0191] Step 102, introducing a function approximation method such as a neural network based on a non-cooperative differential game model to establish a differential game system with unknown parameters;

[0192] In this embodiment, the unknown differential game system is represented as:

[0193]

[0194]

[0195] in, and are the unknown weight matrices, and are activation functions, and is the corresponding approximation error. It can be proved that by increasing the number of neurons and , the approximate error can be converged to an arbitrarily small range, let is the total number of neurons, is the augmented weight matrix, is the augmented activation function vector, is the lumped error, then the unknown differential game system can be expressed as:

[0196]

[0197] Since the true weight coefficient matrix is ​​unknown, the following approximate system is defined:

[0198]

[0199] in, is the estimated value of the weight coefficient matrix.

[0200] Taking the above as an example, select a single hidden layer neural network and select .

[0201] Step 103, adopting an integral parallel learning method, using trajectory data for numerical product identification to identify unknown parameters in the system;

[0202] In this embodiment, W groups are collected The historical trajectory data within is recorded as , and for each set of data, calculate:

[0203]

[0204] in, , define the matrix , the collected historical trajectory data set is required to meet ,The integral parallel learning method simultaneously utilizes the numerical integration of the historical data set and the real-time data to update the unknown parameters in the identification model, thus getting rid of the need for state derivative data in the traditional parallel learning method;

[0205] definition and Respectively represent In the process of iteration, the estimated values ​​of the system state value and coefficient matrix are augmented, and the integral parallel learning method gives the parameter adaptation law of the following form:

[0206]

[0207] in , is the learning rate, It is The estimated error of the system state during the iteration.

[0208] Taking the above example, the control inputs of the driver and the autonomous driving system are randomly given as , , randomly select a time interval A total of 1000 sets of trajectory data are taken as historical trajectory data.

[0209] Step 104, based on the identified differential game system, learn the Nash equilibrium solution in the human-computer game problem through an adaptive iterative learning algorithm.

[0210] In this embodiment, an adaptive iterative algorithm is proposed to solve the human-machine differential game problem under the control input constraint of matrix inequality. The specific steps are as follows:

[0211] S1: Set i=0, initialize coefficients , , , and convergence accuracy , , select the initial control input of the driver and the automatic control system and and the initial coefficient matrix estimates , solve the following set of equations to update the system's initial trajectory estimate and initial weight coefficient matrix estimate:

[0212]

[0213] in, and are the estimation errors of the initial system state trajectory and the initial weight coefficient matrix, The specific expression of is determined by the adaptive law in the integral parallel learning method, is the actual initial state trajectory of the system,

[0214] S2: Calculated by the following formula :

[0215]

[0216] in, Represents the estimated value of the Hamiltonian function.

[0217] S3: Update the driver's control input by solving the following equation:

[0218]

[0219] in, , .

[0220] S4: Calculate the new system state based on the updated control input And the corresponding driver cost function , determine whether the updated driver control input reduces its cost function: if , then let , , is the iteration step length, and returns to S3, otherwise, enters S5.

[0221] S5: Calculate by the following formula :

[0222]

[0223] in, Represents the estimated value of the Hamiltonian function.

[0224] S6: Update the control input of the autonomous driving system by solving the following equation:

[0225]

[0226] in, , .

[0227] S7: Calculate new system state from updated control inputs And the corresponding cost function of the autonomous driving system , determine whether the updated control input of the autonomous driving system reduces its cost function: if , then let , , is the iteration step length, and returns to S6, otherwise, enters S8.

[0228] S8: Update the system's trajectory estimate and weight coefficient matrix estimate by solving the following set of equations:

[0229]

[0230] S9: If , then enter S10, otherwise , , , , And return to S2.

[0231] S10: If , stop the iteration process and get the corresponding Nash equilibrium solution, otherwise let , , , , And return to S2.

[0232] Taking the above as an example, the initial state value of the vehicle system is selected as static: , the initial control inputs for humans and the autonomous driving system are selected as: , , the initial parameters of the adaptive iterative algorithm are set as: , , , the initial coefficient matrix is ​​randomly selected as , as shown below:

[0233]

[0234] Taking the above as an example, different authority allocation coefficients are given, and their Nash equilibrium solutions are calculated through an iterative algorithm.

[0235] Figure 4 Shows different permission allocation coefficients and It can be seen that in each iteration, the control input updates of the driver and the automated system lead to a decrease in their corresponding cost functions and eventually converge. In addition, it can be clearly observed that as the authority of the automated driving system increases, the driver cost function under the Nash equilibrium gradually increases, and the cost function of the autonomous driving system Then it gradually decreases.

[0236] Figure 5 It reflects the comparison between the target trajectories of the driver and the autonomous driving system under different weights. It can be seen that as the authority of the autonomous driving system increases, the actual trajectory of the vehicle under the Nash equilibrium solution is closer to the trajectory planned by the autonomous driving system, and vice versa.

[0237] Figure 6 The Nash equilibrium solutions corresponding to the driver and the autonomous driving system under different weights are shown, and it can be seen that the control input satisfies the given constraints.

[0238] Figure 7-Figure 26 It reflects the comparison between the actual state trajectory of the vehicle and its estimated value under different weights, where Figure 7 , Figure 8 , Fig. 9 , Fig.10 middle , Fig.11 , Fig.12 , Fig.13 , Fig.14 middle , Fig.15 , Fig.16 , Fig.17 , Fig.18 middle , Fig.19 , Fig. 20 , Fig.21 , Fig. 22 middle , Fig.23 , Fig.24 , Fig.25 , Fig.26 middle ,It can be seen that the identification model of the system is very close to the real model of the vehicle.

[0239] Fig. 27 The 3D schematic diagram reflects the change of vehicle trajectory under different weights, which can intuitively observe the change trend of the vehicle's real trajectory with the weight under the Nash equilibrium solution.

[0240] The differential game method for solving the problem of shared control between man and machine in autonomous driving vehicles in the embodiment of the present application establishes a non-cooperative differential game model of the two based on the target trajectory of the driver and the trajectory planned by the autonomous driving system and the control constraints in the form of matrix inequalities; introduces function approximation methods such as neural networks based on the model of non-cooperative differential games to establish a parameter identification model of the unknown game system; uses the integral parallel learning method to perform numerical integration using real-time trajectory data and historical trajectory data to identify unknown parameters in the model; based on the identified model, proposes an adaptive iterative learning algorithm to learn the Nash equilibrium solution in the man-machine game problem. This application takes into account the situation where the control is constrained without relying on the precise parameters of the system model, and improves the control strategy of the driving system by solving the non-cooperative Nash equilibrium solution of man and machine.

[0241] In order to implement the above embodiments, the present application also proposes a differential game device for solving the problem of human-machine shared control of autonomous driving vehicles.

[0242] Fig.28 A schematic diagram of the structure of a differential game device for solving the problem of human-machine shared control of an autonomous driving vehicle provided in an embodiment of the present application.

[0243] like Fig.28 As shown, the differential game device for solving the human-machine shared control problem of the autonomous driving vehicle includes:

[0244] A system building module is used to establish a human-machine non-cooperative differential game model based on the driver's target trajectory and the trajectory planned by the autonomous driving system, taking into account the control constraints in the form of matrix inequalities. The non-cooperative differential game model uses Nash equilibrium to represent the human-machine game problem.

[0245] The system building module is also used to introduce function approximation methods such as neural networks based on the non-cooperative differential game model to establish a differential game system with unknown parameters;

[0246] The system identification module is used to identify unknown parameters in the system by using the integral parallel learning method and the trajectory data for numerical integral identification;

[0247] The computational solution module is used to learn the Nash equilibrium solution in the human-computer game problem through an adaptive iterative learning algorithm based on the identified differential game system.

[0248] It should be noted that the aforementioned explanation of the differential game method embodiment for solving the human-machine shared control problem of an autonomous driving vehicle is also applicable to the differential game device for solving the human-machine shared control problem of an autonomous driving vehicle in this embodiment, and will not be repeated here.

[0249] In order to implement the above embodiments, the present application also proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in the above embodiments is implemented.

[0250] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method of the above embodiments is implemented.

[0251] In order to implement the above embodiments, the present application also proposes a computer program product. When an instruction processor in the computer program product executes, the method described in the above embodiments is executed.

[0252] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0253] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0254] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0255] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0256] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0257] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0258] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0259] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A differential game method for solving the problem of human-machine shared control of an autonomous driving vehicle, characterized in that: include: A non-cooperative differential game model of man-machine is established based on the target trajectory of the driver and the trajectory planned by the automatic driving system and considering the control constraints in the form of matrix inequalities, wherein the non-cooperative differential game model uses Nash equilibrium to represent the man-machine game problem; Based on the non-cooperative differential game model, a neural network function approximation method is introduced to establish a differential game system with unknown parameters; Adopting the integral parallel learning method, the trajectory data is used for numerical product identification of unknown parameters in the system. Based on the identified differential game system, the Nash equilibrium solution in the human-computer game problem is learned through an adaptive iterative learning algorithm; Wherein, establishing the non-cooperative differential game model includes: Build a vehicle control system model; Based on the vehicle control system model, the control inputs of the driver and the automatic driving system are integrated as the shared control inputs of the human-machine shared control system, and a human-machine shared control differential game system is established; Set the permissible control set and cost function for the driver and the autonomous driving system; Assuming that the goals of the driver and the autonomous driving system are to minimize their respective cost functions based on the given differential game system and matrix inequality constraints, and that the driver and the autonomous driving system will not cooperate or negotiate, the game problem is transformed into solving the Nash equilibrium solution based on the assumption, and the necessary conditions satisfied by the game problem are determined.

2. The differential game method for solving the human-machine shared control problem of an autonomous driving vehicle according to claim 1, characterized in that: The vehicle control system model is: in, is the system state variable, is the control input, is the system drift vector function, is the control vector function of the system, is the initial state of the system; The shared control input is: in, , denote the control inputs of the driver and the autonomous driving system, respectively, Assigning coefficients to the authority imposed on the driver and the automated driving system; The human-machine shared control differential game system is: in, is the output variable of the system, is the output matrix of the system; The admissible control set is: in, and are the constraint matrices for the driver and the autonomous driving system, respectively. and are the threshold vectors for the driver and the autonomous driving system, respectively; The cost function is a quadratic cost function, expressed as: in, is the driver’s target trajectory, The target trajectory planned by the autonomous driving system, and are the output tracking error weight matrices for the driver and the autonomous driving system, and are the control energy weight matrices for the driver and the autonomous driving system, For a given finite time domain; The game problem is transformed into a solution to the Nash equilibrium solution, and the formula is: The necessary conditions satisfied by the game problem include a first set of equations and a second set of equations, wherein the first set of equations is: The second set of equations is: in, and are the co-state quantities of the driver and the automatic driving system, and are the Hamiltonian functions of the driver and the autonomous driving system respectively, expressed as; 。 3. The differential game method for solving the human-machine shared control problem of an autonomous driving vehicle as claimed in claim 2, characterized in that: Establishing a differential game system of the unknown parameters, including: The unknown differential game system is expressed as: in, and is the unknown weight matrix, and is the activation function, and is the corresponding approximation error; make is the total number of neurons, is the augmented weight matrix, is the augmented activation function vector, , is the lumped error, and the unknown differential game system is expressed as: When the true weight system matrix is ​​unknown, the approximate system is defined as: in, is the estimated value of the weight coefficient matrix.

4. The differential game method for solving the human-machine shared control problem of an autonomous driving vehicle as claimed in claim 3, characterized in that: Identify unknown parameters in the system, including: Collect W group The historical trajectory data in is recorded as , and for each set of data, calculate: in, ; Defining the Matrix , the collected historical trajectory data set meets , the integral parallel learning method is used to simultaneously use the numerical integration of the historical data set and the real-time data to update the unknown parameters in the differential game system. When updating, the parameter adaptation law is: in, , is the learning rate, is the estimated error of the system state during the i-th iteration.

5. The differential game method for solving the human-machine shared control problem of an autonomous driving vehicle as claimed in claim 4, characterized in that: Study the Nash equilibrium solutions in human-computer game problems, including: Step S1: Set i=0; initialize coefficients , , , and convergence accuracy , , select the initial control input of the driver and the automatic control system and and the initial coefficient matrix estimates , solve the following set of equations to update the system's initial trajectory estimate and initial weight coefficient matrix estimate: in, and are the estimation errors of the initial system state trajectory and the initial weight coefficient matrix, The specific expression of is determined by the adaptive law in the integral parallel learning method, is the actual initial state trajectory of the system; Step S2: Calculate by the following formula : in, represents the estimated value of the Hamiltonian function; Step S3: Update the driver's control input by solving the following equation: in, , ; Step S4: Calculate the new system state through the updated control input And the corresponding driver cost function , determine whether the updated driver control input reduces its cost function. If , then let , , is the iteration step, return to step S2, otherwise, go to step S5; Step S5: Calculate by the following formula : in, represents the estimated value of the Hamiltonian function; Step S6: Update the control input of the autonomous driving system by solving the following equation: in, , ; Step S7: Calculate the new system state through the updated control input And the corresponding cost function of the autonomous driving system , to determine whether the updated control input of the autonomous driving system reduces its cost function. If , then let , , is the iteration step, return to step S6, otherwise, go to step S8; Step S8: Update the system trajectory estimate and weight coefficient matrix estimate by solving the following equations: Step S9: If , then go to step S10, otherwise, let , , , , , and return to step S2; Step S10: If , stop the iteration process and get the corresponding Nash equilibrium solution, otherwise, let , , , , , and return to step S2.

6. A differential game device for solving the problem of human-machine shared control of an autonomous driving vehicle, characterized in that: include: A system establishment module, used to establish a human-machine non-cooperative differential game model based on the driver's target trajectory and the trajectory planned by the automatic driving system, taking into account the control constraints in the form of matrix inequalities, wherein the non-cooperative differential game model uses Nash equilibrium to represent the human-machine game problem; The system establishment module is also used to introduce a neural network function approximation method based on the non-cooperative differential game model to establish a differential game system with unknown parameters; The system identification module is used to identify unknown parameters in the system by using the integral parallel learning method and the trajectory data for numerical integral identification; The computational solution module is used to learn the Nash equilibrium solution in the human-computer game problem through an adaptive iterative learning algorithm based on the identified differential game system; Wherein, establishing the non-cooperative differential game model includes: Build a vehicle control system model; Based on the vehicle control system model, the control inputs of the driver and the automatic driving system are integrated as the shared control inputs of the human-machine shared control system, and a human-machine shared control differential game system is established; Set the permissible control set and cost function for the driver and the autonomous driving system; Assuming that the goals of the driver and the autonomous driving system are to minimize their respective cost functions based on the given differential game system and matrix inequality constraints, and that the driver and the autonomous driving system will not cooperate or negotiate, the game problem is transformed into solving the Nash equilibrium solution based on the assumption, and the necessary conditions satisfied by the game problem are determined.

7. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the differential game method for solving the human-machine shared control problem of an autonomous driving vehicle as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the differential game method for solving the human-machine shared control problem of an autonomous driving vehicle as described in any one of claims 1-5.

9. A computer program product, when the instructions in the computer program product are executed by a processor, executes the differential game method for solving the human-machine shared control problem of an autonomous driving vehicle as described in any one of claims 1 to 5.

Images