Game cooperative control method and device for man-machine co-driving type intelligent automobile

Abstract

The invention discloses a game cooperative control method and device for a man-machine co-driving type intelligent automobile. The method comprises the steps that a man-machine path control game model corresponding to man-machine interaction behaviors of a driver and an automatic system is established; solving a man-machine path control game model based on the approximate dynamic programming framework model to obtain a target control combination of a driver and an automatic system; wherein the target control combination comprises target control inputs corresponding to the driver and the automation system; the target control input enables revenue functions corresponding to the driver and the automation system to be minimum; the revenue function is used for indicating the deviation between the driving path of the vehicle and the expected path; and performing vehicle control according to the target control combination of the driver and the automation system. The method aims at realistically simulating a man-machine co-driving model, reducing man-machine conflicts and improving man-machine cooperation performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a game-based collaborative control method and device for a human-machine co-driving intelligent vehicle. Background Art

[0002] Human-machine collaborative control means that the driver and the intelligent control system are in the loop at the same time, share control of the vehicle, and complete driving tasks collaboratively. It can combine the advantages of both humans and machines to build a human-machine collaborative hybrid intelligent system.

[0003] At present, most of the human-machine cooperative hybrid intelligent systems based on non-cooperative games are based on the assumption of complete information and are usually divided into two types: Nash equilibrium and Stackelberg equilibrium. Nash equilibrium means that under the given control strategy of one party, the optimal strategy is selected, and the equilibrium state is achieved when both parties reach their respective minimum functions; in Stackelberg equilibrium, the driver usually has a better understanding of the control strategy of the vehicle system, and the autonomous driving system can estimate and correct the driver's driving plan to achieve a balance between the two parties.

[0004] However, in reality, drivers and vehicle automation systems are not able to fully grasp each other's benefits and other information, so the current research field is still too idealistic. Summary of the invention

[0005] The present invention provides a game cooperative control method and device for a human-machine co-driving intelligent car; the method aims to create a realistic human-machine co-driving model, reduce human-machine conflicts and improve human-machine cooperative performance.

[0006] To achieve the above-mentioned purpose, according to a first aspect of an embodiment of the present application, a game cooperative control method for a human-machine co-driving intelligent vehicle is provided, the method comprising: Establishing a human-machine path control game model corresponding to the human-machine interaction behavior between the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; forming two candidate control combinations of the driver and the automation system based on the two expected paths of the driver and the expected path of the automation system; Based on the approximate dynamic programming framework model, the human-machine path control game model is solved to obtain the target control combination of the driver and the automation system; wherein the target control combination includes the target control inputs corresponding to the driver and the automation system respectively; the target control inputs minimize the benefit functions corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation of the vehicle's driving path from the expected path; Vehicle control is performed based on a combination of control objectives of the driver and the automation system.

[0007] In a possible implementation, establishing a human-machine path control game model corresponding to the human-machine interaction behavior between the driver and the automation system includes: Based on the indirect shared control mode between the driver and the automation system, a mathematical model of the control input corresponding to the candidate control combination of the driver and the automation system is constructed; wherein the mathematical model of the control input is used to indicate the control input of the driver and the control input of the automation system in the candidate control combination; Based on the control input mathematical model corresponding to the candidate control combination and the pre-constructed vehicle dynamics model, constructing the benefit functions corresponding to the driver and the automation system in the candidate control combination respectively; wherein the vehicle dynamics model is an expression for indicating the vehicle state of the vehicle; The human-machine path control game model is obtained based on the respective corresponding benefit functions of the driver and the automation system in the two groups of candidate control combinations.

[0008] In a possible implementation manner, the constructing of a mathematical model of control inputs corresponding to the candidate control combination of the driver and the automation system based on the indirect shared control mode between the driver and the automation system includes: The value of the driver's guess about the expected path selected by the automated system is always 1, and the probability that the automated system guesses that the driver will select one of the two candidate expected paths is , then the probability that the driver chooses the other one is , the vehicle's control input is the control input of the driver and the automation system respectively , The weighted sum of , and are all positive numbers not greater than 1; without considering noise, the control input of the vehicle corresponding to each candidate control combination is expressed as: , In the formula, represents the control input of the vehicle, A weighting factor for the driver's control input; k Represents a time series of samples.

[0009] In a possible implementation, the method further includes constructing a dynamic model of the vehicle, wherein constructing the dynamic model includes: Based on the preset model constraint parameters, a two-degree-of-freedom dynamic model of the vehicle is constructed, wherein the model constraint parameters include: the longitudinal speed of the vehicle is a constant value, the front wheel steering angle of the vehicle satisfies that the tire works in the linear region, and the heading angle deviation satisfies that the distance between the heading of the vehicle and the center line is not greater than a preset value; the two-degree-of-freedom dynamic model of the vehicle is expressed as follows: , In the formula, Represents the vehicle state. In the vehicle state expression, represents the two-dimensional ordinate position of the vehicle, represents the lateral velocity of the vehicle, represents the vehicle yaw angle in two-dimensional global coordinates, represents the angular velocity, Indicates the vehicle running speed. Indicates the two-dimensional horizontal coordinate position of the vehicle; Indicates the vehicle status Find the differential; represents the observed value; represents the control input, the weight of the turning angle; represents the model error; represents the observation noise; represents the observation matrix; The differential formula of the state variable is as follows, which is constructed without considering the model error. and The specific form is: , In the formula, Indicates the two-dimensional vertical coordinate position of the vehicle The derivative of Represents the vehicle's yaw angle in two-dimensional global coordinates The derivative of Represents the two-dimensional horizontal coordinate position of the vehicle The derivative of Indicates the vehicle running speed The derivative of Indicates the lateral speed of the vehicle The derivative of , are the front and rear cornering stiffness, is the vehicle mass, is the polar moment of inertia, is the constant steering ratio, is the longitudinal acceleration, Represent the axial distance from the vehicle's center of mass to the front and rear wheels respectively.

[0010] The vehicle state of the two-degree-of-freedom dynamics model of the vehicle is discretized using the forward Euler method to obtain a vehicle dynamics model with discretized sampling time, including: The vehicle dynamics model with discretized sampling time is expressed as follows: , In the formula, Indicates the time when sampling is performed, Indicates the sampling time point sequence number.

[0011] In a possible implementation, constructing the benefit functions corresponding to the driver and the automation system in the candidate control combination based on the control input mathematical model corresponding to the candidate control combination and the pre-built vehicle dynamics model includes: Based on the control input of the vehicle corresponding to the candidate control combination and the pre-built vehicle dynamics model, the vehicle state of the vehicle is obtained, and the expression of the vehicle state is as follows: ; In the formula, Indicates the vehicle status. Indicates vehicle status The derivative of is the weighting coefficient of the driver's control input, represents the driver's control input; represents the control input of the automation system; represents the model error; Based on the vehicle state of the vehicle under the control combination, the control strategy of the driver on the vehicle and the control strategy of the automation system on the vehicle, the benefit functions corresponding to the driver and the automation system in the candidate control combination are constructed; wherein the benefit functions corresponding to the driver and the automation system in the candidate control combination are expressed as: , In the formula, is the corresponding benefit function of the driver in the candidate control combination; is the profit function corresponding to the automation system in the candidate control combination; in, , , In the formula, represents the observed value of the vehicle in the candidate control combination, and represent the target observations of the driver and the automation system respectively; is the loss function corresponding to the driver; is the loss function corresponding to the automation system, and They are all second-order positive definite matrices used to adjust the control ratio; The payoff functions of the driver and the automation system satisfy the Bayesian Nash equilibrium of the incomplete information game. , In the formula, represents the control strategy of the automation system, represents the optimal target control strategy of the automation system; and They represent the driver’s control strategies under two desired paths, and They represent the driver’s optimal control strategies under the two desired paths, and represents the driver’s profit function under two expected paths, and represents the profit function of the automation system under two expected paths, p Indicates the inferred probability.

[0012] In a possible implementation, solving the human-machine path control game model based on the approximate dynamic programming framework model to obtain the target control combination of the driver and the automation system includes: For each of the candidate control combinations, construct an approximate dynamic programming framework model corresponding to each of the candidate control combinations; wherein the approximate dynamic programming framework model includes prediction networks corresponding to the driver and the automation system, respectively, and the prediction network includes a strategy network and an evaluation network; For each of the candidate control combinations, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system, respectively, and the prediction network corresponding to the driver outputs a first current profit function corresponding to the current state of the vehicle and a first next profit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current profit function corresponding to the current state of the vehicle and a second next profit function corresponding to the next state of the vehicle; For each of the candidate control combinations, based on a first next benefit function output by the prediction network corresponding to the driver and a first current loss function of the driver in a current state of the vehicle, updating a policy network of the prediction network corresponding to the driver, and based on the first next benefit function output by the prediction network corresponding to the driver and a first current benefit function corresponding to the current state of the vehicle, updating an evaluation network of the prediction network corresponding to the driver; Based on the second benefit function corresponding to the next vehicle state output by each of the two candidate control combinations and the second current loss function of the automation system in the current vehicle state, the evaluation network of the prediction network corresponding to the automation system is updated.

[0013] In a possible implementation, for each of the candidate control combinations, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system, respectively, the prediction network corresponding to the driver outputs a first current benefit function corresponding to the current state of the vehicle and a first next benefit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current benefit function corresponding to the current state of the vehicle and a second next benefit function corresponding to the next state of the vehicle, including: Inputting the current vehicle state of the vehicle into a policy network corresponding to the driver and outputting a first current control input, and inputting the current vehicle state into a policy network corresponding to the automation system and outputting a second current control input; Based on the first current control input and the second current control input corresponding to each of the candidate control combinations, the next vehicle state of the vehicle is predicted, the predicted next vehicle state is input into the evaluation network corresponding to the driver, and a first benefit function corresponding to the next vehicle state of the vehicle is output; and the predicted next vehicle state is input into the evaluation network corresponding to the automation system, and a second benefit function corresponding to the next vehicle state of the vehicle is output.

[0014] For each of the candidate control combinations, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system, respectively, the prediction network corresponding to the driver outputs a first current benefit function corresponding to the current state of the vehicle and a first next benefit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current benefit function corresponding to the current state of the vehicle and a second next benefit function corresponding to the next state of the vehicle, including: Inputting the current vehicle state of the vehicle into a policy network corresponding to the driver and outputting a first current control input, and inputting the current vehicle state into a policy network corresponding to the automation system and outputting a second current control input; Based on the first current control input and the second current control input corresponding to each of the candidate control combinations, the next vehicle state of the vehicle is predicted, the predicted next vehicle state is input into the evaluation network corresponding to the driver, and a first benefit function corresponding to the next vehicle state of the vehicle is output; and the predicted next vehicle state is input into the evaluation network corresponding to the automation system, and a second benefit function corresponding to the next vehicle state of the vehicle is output.

[0015] To achieve the above-mentioned purpose, according to a second aspect of an embodiment of the present application, a game cooperative control device for a human-machine co-driving intelligent vehicle is further provided, the device comprising: A modeling module is used to establish a human-machine path control game model corresponding to the human-machine interaction behavior of the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; based on the two expected paths of the driver and the expected path of the automation system, two candidate control combinations of the driver and the automation system are formed; A solution module, for solving the human-machine path control game model based on an approximate dynamic programming framework model, and obtaining a target control combination of the driver and the automation system; wherein the target control combination includes target control inputs corresponding to the driver and the automation system respectively; the target control input minimizes the benefit function corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation of the vehicle's driving path from the expected path; The application module is used for controlling the vehicle according to the target control combination of the driver and the automation system.

[0016] To achieve the above-mentioned purpose, according to the third aspect of an embodiment of the present application, a processor is also provided; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in the first aspect.

[0017] To achieve the above-mentioned purpose, according to the fourth aspect of the embodiment of the present application, a computer-readable medium is further provided, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0018] Compared with the prior art, an embodiment of the present invention provides a game-based collaborative control method and device for a human-machine co-driving intelligent vehicle, which includes: first, establishing a human-machine path control game model corresponding to the human-machine interaction behavior of the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; two candidate control combinations of the driver and the automation system are formed based on the two expected paths of the driver and the expected path of the automation system; secondly, based on an approximate dynamic programming framework model, the human-machine path control game model is solved to obtain a target control combination of the driver and the automation system; wherein the target control combination includes the target control inputs corresponding to the driver and the automation system respectively; the target control input minimizes the benefit functions corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation between the vehicle's driving path and the expected path; finally, the vehicle is controlled according to the target control combination of the driver and the automation system. Therefore, in the complex game decision-making process in the traffic control system where humans and autonomous driving coexist, dynamic game benefits are introduced as feedback to resolve the possible conflict between individual local optimal strategies and collective global optimal benefits in group decision-making; the predictive reasoning of game theory and the convergence and stability analysis of control theory are flexibly used to take into account the uncertainty in the learning and decision-making process to the greatest extent, thereby promoting the establishment of a theoretical framework with stochastic games and stochastic control as the core. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application; Figure 2 A direct sharing control mode provided in an embodiment of the present application; Figure 3 A flowchart of the game-based collaborative control method for a human-machine co-driving intelligent vehicle provided in this application; Figure 4 The two-degree-of-freedom vehicle dynamics model of the game-based collaborative control method for the human-machine co-driving intelligent vehicle provided in this application; Figure 5 The indirect shared control structure of the game-based collaborative control method for the human-machine co-driving intelligent vehicle provided in this application; Figure 6The cross-iteration method of two sets of dual ADP frameworks of the game cooperative control method of the human-machine co-driving intelligent car provided in this application; Figure 7 A dual iterative ADP framework for the game-themed collaborative control method of a human-machine co-driving intelligent vehicle provided in this application; Figure 8 A combined iterative ADP framework for the game-theoretic collaborative control method of a human-machine co-driving intelligent vehicle provided in this application; Fig. 9 A schematic structural diagram of a game-based collaborative control method device for a human-machine co-driving intelligent vehicle provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] Figure 1 The following is a schematic diagram of an application scenario provided by an embodiment of the present application, such as Figure 1 As shown in the figure, the basic framework of human-machine collaborative control consists of three stages: perception layer, decision layer and execution layer. Among them, the perception layer refers to the perception of the traffic environment by both humans and machines. The system mainly uses advanced vehicle-mounted sensors to obtain environmental information and transmit it to the driver; the decision layer solves the expected decision and trajectory based on the perception information; the execution layer generates control instructions that actually take effect.

[0023] like Figure 2 As shown in the figure, indirect shared control, namely visual shared control, is a shared control scheme that only uses the visual steering characteristics of human drivers to control the steering wheel angle. The steering wheel angle is converted into torque after calculation by the intermediate controller, and will not directly interfere with the control program of the automation system. Compared with direct (tactile) shared control, this non-invasive fusion control can effectively reduce the human-machine conflict between the intelligent system and the driver, thereby improving the stability of vehicle driving. Compared with direct shared control, indirect shared control has greater development potential in terms of design flexibility and anti-intrusion, because its risk protection mechanism is safer and more reliable. At the same time, its steer-by-wire chassis technology and active steering mode are more in line with the future development trend of steering systems, so it has better research value and application prospects.

[0024] The present invention constructs a static Bayesian Nash equilibrium shared controller for improving lane keeping performance. An indirect shared control framework that can integrate the perception, decision-making and control authority of the driver and the automation system is adopted. In this mode, the control input of the intelligent car is the weighted sum of the driver and the automation system; secondly, an incomplete information controller is designed, and a practical incomplete information game assumption is made on the basis of the traditional complete information game theory, and the optimal strategy expression when the automation system reaches equilibrium under the static Bayesian game theory is derived.

[0025] Figure 3 Schematic diagram of the process of the game collaborative control method for the human-machine co-driving intelligent car provided in this application Figure 1 ,like Figure 3 As shown, the method includes: S301. Establish a human-machine path control game model corresponding to the human-machine interaction behavior between the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; two candidate control combinations of the driver and the automation system are formed based on the two expected paths of the driver and the expected path of the automation system.

[0026] Exemplarily, the execution subject of this embodiment may be an electronic device, or a terminal device, or a server, or a device or equipment that can execute the solution of this embodiment, and there is no limitation on this.

[0027] It is understandable that for the lane keeping problem, the driver and the automated system have their own benefits, which is a variable related to the system state and control input in optimal control theory. In the real world, incomplete information is inevitable. Therefore, based on the traditional complete information game, the driver's benefits are considered as information that only the driver can know, and the vehicle control problem is described as a static Bayesian game problem of incomplete information game: The two parties involved in this game problem are the automatic system (abbreviated as A in this invention) and the human driver (abbreviated as D in this invention). Taking action in control input , jointly control the movement of the vehicle, here the action space A collection of control inputs (values) used to indicate possible actions. Used to indicate the vehicle's trajectory, such as the vehicle's steering wheel angle.

[0028] Automated systems and human drivers each have their own type space This type of space refers to the target lane that meets the respective expectations. For the automated system A, its target lane only contains the preset path. ,Right now , which is common information known to both humans and the system; the human driver D will have two different desired lanes ,Right now ; People and cars want to move along the center line, so they deviate from the road to the left or right.

[0029] The target trajectory of automation system A is a preset value. This type of space contains only one variable. is an inevitable event, so driver D, knowing his own target trajectory, can make a guess about the type selected by A. The value of is always 1, so this quantity can be ignored. Similarly, suppose that the automatic control system A knows the type it has chosen, and infers that D chooses The probability of , then it is inferred that D chooses The probability of ; When different control strategies are adopted, the automated system A and the human driver D will obtain their respective benefit functions , since driver D has two possible desired paths, there is a corresponding benefit function for each possible path .

[0030] In an example, the above step S301 may include: Based on the indirect shared control mode between the driver and the automation system, a mathematical model of control input corresponding to the candidate control combination of the driver and the automation system is constructed, wherein the mathematical model of control input is used to indicate the control input of the driver and the control input of the automation system in the candidate control combination.

[0031] Based on the control input mathematical model corresponding to the candidate control combination and the pre-built vehicle dynamics model, the benefit functions corresponding to the driver and the automation system in the candidate control combination are constructed, wherein the vehicle dynamics model is an expression for indicating the vehicle state of the vehicle.

[0032] Based on the corresponding profit functions of the driver and the automation system in the two sets of candidate control combinations, the human-machine path control game model is obtained.

[0033] In one example, the mathematical model of control input corresponding to the candidate control combination of the driver and the automation system based on the indirect shared control mode between the driver and the automation system may include: The value of the driver's guess about the desired path selected by the automated system is always 1, and the probability that the automated system guesses that the driver will choose one of the two candidate desired paths is , then the probability that the driver chooses the other one is , the vehicle's control input is the control input of the driver and the automation system respectively The weighted sum of , and are all positive numbers not greater than 1. Without considering noise, the control input of the vehicle corresponding to each candidate control combination is expressed as: , In the formula, represents the control input of the vehicle, A weighting factor for the driver's control input; k Indicates the sampling time point sequence number.

[0034] It is understood that the vehicle inputs for indirect shared control are the inputs from the driver and the automated system. The weighted sum of When the smart car input , the vehicle is completely controlled by the driver. When input , the vehicle is completely controlled by the automation system. This control structure allows the driver and the automation system to be in the loop at the same time, which not only avoids certain offline risks, but also expands the range of control situations that the automation system can handle. At the same time, its simple control form can be implemented in engineering using existing algorithms related to automatic control principles, thus having obvious advantages.

[0035] For example, Figure 4 The two-degree-of-freedom dynamics model of the vehicle for the game cooperative control method of the human-machine co-driving intelligent vehicle provided in this application is established. A two-degree-of-freedom dynamics model of the vehicle lane keeping operation under the global coordinate XOY is established, such as Figure 4 As shown in the figure, the symbols in the figure represent the following meanings: : The center of mass of the vehicle; : The distance from the center of mass to the front axle; : The distance from the center of mass to the rear axle; : longitudinal speed; : lateral displacement deviation; : vehicle yaw angle; : The expected yaw angle of the vehicle at the current moment; : Front wheel steering angle.

[0036] In order to make the model more accurate and specific, the following assumptions are made here: 1) The vehicle longitudinal velocity is a constant value; 2) the vehicle's front wheel steering angle Small, that is, the tire works in the linear region; 3) Heading angle deviation Small and no side slip, that is, the vehicle's heading is near the center line. Then the two-degree-of-freedom dynamic model of the vehicle in continuous time can be expressed as: , That Represents the vehicle state. In the vehicle state expression, represents the two-dimensional ordinate position of the vehicle, represents the lateral velocity of the vehicle, represents the vehicle yaw angle in two-dimensional global coordinates, represents the angular velocity, Indicates the vehicle running speed. Indicates the two-dimensional horizontal coordinate position of the vehicle; Indicates the vehicle status Find the differential, Represents the observed value, model input represents the weight of the corner, and Represents model error and observation noise, the observation matrix for , The differential formula of the state variable is as follows, which is constructed without considering the model error. and The specific form is: , In the formula, Indicates the two-dimensional vertical coordinate position of the vehicle The derivative of Represents the vehicle's yaw angle in two-dimensional global coordinates The derivative of Represents the two-dimensional horizontal coordinate position of the vehicle The derivative of Indicates the vehicle running speed The derivative of Indicates the lateral speed of the vehicle The derivative of are the front and rear cornering stiffness, is the vehicle mass, is the polar moment of inertia, is the constant steering ratio, is the longitudinal acceleration, which is controlled by the PID controller to keep the longitudinal speed constant. Represent the axial distance from the vehicle's center of mass to the front and rear wheels respectively.

[0037] In practical applications, the forward Euler method is used to discretize the state of the vehicle dynamics model: , In the formula Indicates the time when sampling is performed, It represents the serial number of the sampling time point. After modeling the entire vehicle system, the incomplete information game controller can be designed based on the model to solve the optimal control problem of vehicle trajectory tracking.

[0038] It can be understood that under the vehicle dynamics model system, the vehicle state expression after indirect shared control can be decomposed into the following form: , Due to the vehicle control strategy Only the current motion state of the smart car (vehicle state) Therefore, the dynamic model of the vehicle can also be written as follows: , in, , Therefore, each participant (driver / automated system) can regard the converted dynamic model and its payoff function as an optimal control problem and solve it iteratively through adaptive dynamic programming theory.

[0039] It is understandable that conflict and cooperation are ultimately reflected in different expected trajectories (expected paths), forming a non-zero-sum differential game between the driver and the automated system. Considering that the driver has two possible expected paths, there are two possible combinations. Under this condition, the driver and the automation system have their own benefit functions , the profit function with the expected path can be expressed as: , in, , and represents the vehicle state observations of the vehicle in two possible situations, Indicates two different trajectories 1 and 2 that can be selected. According to the vehicle dynamics model, the constant matrix (matrix constant) of the vehicle controller is set and It can be used to balance error and control energy. Both the driver and the automated system try to minimize their own reward function and In this case, the driver’s expectations are manually constructed based on existing research results, Since the automated system needs to simulate the driver taking two different target trajectories when making a strategy choice, its optimal benefit function is an average expectation related to the observation probability, and the optimal strategy is the value that makes the following equation true: , In the formula and They represent the control strategy of the automation system and the control strategy of the driver under the two target paths respectively. Indicates the control strategy adopted in the automation system Next, the driver selects As the optimal control strategy (target control strategy) for the target trajectory, add Indicates the best, and Similarly, it means that the control strategy is adopted in the automatic system Next, the driver selects As the optimal control strategy for the target trajectory (target control strategy), the control strategy Combined vehicle status Control input can be obtained , then the input of each time instance can be expressed as: , Since the driver and the automated system are assumed to pursue their respective desired paths, a Bayesian Nash equilibrium can be obtained for this non-zero-sum differential game, with the following inequality: The Bayesian Nash equilibrium for incomplete information games is satisfied, , In the formula, represents the control strategy of the automation system, Represents the target control strategy of the automation system; and They represent the driver’s control strategies under two desired paths, and They represent the driver’s optimal control strategies under the two desired paths, p Indicates the inferred probability.

[0040] This equilibrium means that the driver's control strategy in each target lane trajectory should minimize the reward function of the trajectory, and the optimal control strategy adopted by the automated system is to minimize the average expected value of the reward function when the driver adopts the same strategy in two possible trajectories. There can only be one, , then the driver’s strategy , the control strategy combination of the intelligent vehicle when reaching equilibrium is .

[0041] It is understandable that the automated system knows its own corresponding target lane and benefit function, and does not need to consider too many complex driver choices, except that it is known that the driver chooses between two lanes and the probability sum is 1. This helps the automated system to make decisions quickly based on the relevant parameters of its own target lane, such as adjusting vehicle speed, vehicle distance and other parameters more quickly in traffic flow control, reducing calculations and decision-making time. The automated system operates according to its own target lane and benefit function, which can more effectively allocate the vehicle's power, braking and other resources.

[0042] In this embodiment, type spaces are set for the automated system and the human driver respectively. For example, the setting of public information that the target lane of the automated system is a preset path, and the setting that the human driver has two desired lanes, reflects the different characteristics of the two in the target lane selection. This setting is both in line with the actual situation and helps to conduct in-depth analysis of the problem under the framework of incomplete information game. The probability of the two parties' speculation on each other's type is considered in the analysis, such as the driver's speculation value of the type selected by the automated system is always 1, and the automated system's probability speculation on the driver's selection of different target trajectories. This makes the model closer to the situation of incomplete information in reality, because in the actual lane control scenario, the two parties cannot fully determine the intention of the other party, and this probabilistic speculation increases the authenticity and reliability of the model. Since the driver has two possible expected paths, a corresponding benefit function is set for each possible path. This reflects the full consideration of the diversity of driver decisions, because different expected paths will lead to different benefits for the driver under different control strategies, which helps to comprehensively analyze the driver's decision motivation and optimal strategy selection in the lane keeping problem.

[0043] S302. Based on the approximate dynamic programming framework model, the human-machine path control game model is solved to obtain the target control combination of the driver and the automation system; wherein the target control combination includes the target control inputs corresponding to the driver and the automation system respectively; the target control inputs minimize the benefit functions corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation of the vehicle's driving path from the expected path.

[0044] It is understandable that if Figure 6As shown, in this embodiment, two sets of dual iterative heuristic dynamic programming frameworks are used to solve the optimal control strategies of both the driver and the autonomous driving system under the incomplete information game. The incomplete information game after the Harsanyi transformation is transformed into a complete but imperfect information game. Assume that in the incomplete information game, the driver has two possible lanes (expected paths). The two sets of frameworks (two combinations) correspond to the two possible types of human drivers, namely lane 1 and lane 2; each lane has a strategy for the driver and the autonomous driving system belonging to the lane. and And the benefit function of both and After clarifying the iteration method within each group, the two groups of dual ADP frameworks are combined, and the cross iteration method is as follows: Figure 6 As shown, each dotted line circle contains a lane, that is, the driver's desired path and the automated system's desired path constitute a candidate control combination of the driver and the automated system. and They represent the driver’s control strategies under the two target lane paths, and represents the driver’s profit function under two expected paths, and Represents the profit function of the automation system under two expected paths.

[0045] In one example, the above step 302 may include: For each candidate control combination, an approximate dynamic programming framework model corresponding to each candidate control combination is constructed; wherein the approximate dynamic programming framework model includes prediction networks corresponding to the driver and the automation system respectively, and the prediction network includes a strategy network and an evaluation network; For each candidate control combination, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system respectively, and the prediction network corresponding to the driver outputs a first current profit function corresponding to the current state of the vehicle and a first next profit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current profit function corresponding to the current state of the vehicle and a second next profit function corresponding to the next state of the vehicle; For each candidate control combination, based on a first next benefit function output by the prediction network corresponding to the driver and a first current loss function of the driver in a current state of the vehicle, a policy network of the prediction network corresponding to the driver is updated, and based on the first next benefit function output by the prediction network corresponding to the driver and a first current benefit function corresponding to the current state of the vehicle, an evaluation network of the prediction network corresponding to the driver is updated; Based on the second benefit function corresponding to the next vehicle state output by each of the two candidate control combinations and the second current loss function of the automation system in the current vehicle state, the evaluation network of the prediction network corresponding to the automation system is updated.

[0046] Repeat the above steps until the optimal profit function is found.

[0047] Understandably, the choice was to use an Actor-Critic (AC) Approximate Dynamic Programming (ADP) framework to solve the system of equations via a numerical iterative algorithm, also known as Heuristic Dynamic Programming (HDP). The AC-ADP framework uses a neural network (NN) to approximate the reward function and And the policy function and Here, the controller is modeled using a heuristic dynamic programming (HDP) structure to estimate the benefit function That is, the output of the evaluation network is used to estimate the profit function, and the parameters of the evaluation network are based on the target value Update, the parameters of the policy network are updated by minimizing the Hamilton (HJB) equation to update.

[0048] like Figure 7 and 8 As shown in Figure 1, based on the above ADP structure, two sets of dual iterative ADP frameworks are proposed to model and solve the Bayesian Nash equilibrium of non-zero-sum differential games. In this dual iterative ADP framework, both the driver and the automation system are modeled based on the AC framework, which has the ability of self-learning and self-evolution. The Bayesian Nash equilibrium of this game refers to the best countermeasure strategy for each other under rational circumstances. Each set of framework structures is as follows: Figure 7 As shown, the combined frame structure is as follows Figure 8 shown.

[0049] Exemplarily, for each candidate control combination, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system, respectively, the prediction network corresponding to the driver outputs a first current benefit function corresponding to the current state of the vehicle and a first next benefit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current benefit function corresponding to the current state of the vehicle and a second next benefit function corresponding to the next state of the vehicle, which may include: Inputting the current vehicle state of the vehicle into a policy network corresponding to the driver and outputting a first current control input, and inputting the current vehicle state into a policy network corresponding to the automation system and outputting a second current control input; Based on the first current control input and the second current control input corresponding to each candidate control combination, the next vehicle state of the vehicle is predicted, the predicted next vehicle state is input into the evaluation network corresponding to the driver, and the first benefit function corresponding to the next vehicle state of the vehicle is output; and the predicted next vehicle state is input into the evaluation network corresponding to the automation system, and the second benefit function corresponding to the next vehicle state of the vehicle is output.

[0050] Exemplarily, the driver and the automation system include a policy network and an evaluation network. By iteratively updating each of their evaluation networks and policy networks, the human-machine path control game model is solved to obtain the target control combination of the driver and the automation system. V is used to represent the cost of the vehicle taking the current strategy in the current state, and is used to evaluate the strategy taken. The benefit function is a function used to describe the gap between the current state of the vehicle and the target state, which is used to evaluate the quality of the vehicle state and estimate the value function. and The self-consistency condition can be expressed as: , According to the structure of the split dynamics model, each value function should satisfy the Bellman equation, , in and Indicates that when the target trajectory is When , the optimal value function of the automation system and the driver is and Indicates that when the target trajectory is When , the optimal value function of the automation system and the driver. Then under the Bayesian Nash equilibrium condition, the optimal strategy of the two participants can be solved as: , Always Select is the actual trajectory of the driver, that is , the actual running status of the vehicle , and because the strategy of the automation system is the result of comprehensive consideration of two possible combinations, and the output under the two combinations is the same , therefore, the optimal strategy for the final operation of the vehicle can be expressed as: , In the two sets of dual iterative ADP frameworks proposed in this paper, the value function and the strategy function are both approximated by neural networks and parameterized as and , and the other contains the parameters ,in and Define policies for network parameters. Sum value function In Status The cumulative future cost under is: , Strategy Sum value function In Status The cumulative future cost under , in is the discount factor, and the strategy evaluation process of the dual iterative ADP is performed by iteratively minimizing the following loss function: , In the policy improvement step, the parameters of the policy network are updated and Minimize the objective function: , It is understandable that ADP combines the core concept of dynamic programming with the approximate technology of machine learning, and can realize the optimization control strategy in complex systems, especially for those problems with huge state space or that cannot be solved accurately. In the present invention, ADP is used to solve the optimal control problem of nonlinear systems, and the iterative value function and iterative control law are approximated through function approximation structures such as neural networks, thereby effectively solving the "curse of dimensionality" problem of traditional dynamic programming methods. Two sets of dual iterative heuristic dynamic programming frameworks: The invention proposes two sets of dual iterative ADP frameworks for modeling and solving Bayesian Nash equilibrium of non-zero-sum differential games. Each set of frameworks corresponds to two possible types of human drivers (lane 1 and lane 2), and each lane has a strategy and value function for the driver and the autonomous driving system belonging to the lane. Each participant (driver and automated system) includes a policy network and a value network, and its value network and policy network are updated by their respective policy functions, including policy evaluation and policy improvement processes. By solving the coupled Hamilton-Jacobi-Bellman (HJB) equations of discrete-time systems, the Bayesian Nash equilibrium solution of differential games can be obtained. Since the system has multi-dimensional state variables and multi-dimensional inputs, the AC-ADP framework (i.e., heuristic dynamic programming HDP) is used to solve the system of equations through a numerical iterative algorithm. In the policy evaluation process, the specific loss function is iteratively minimized; in the policy improvement step, the parameters of the policy network are updated to minimize the objective function (revenue function), and the value and policy networks can be updated through optimization methods such as stochastic gradient descent (SGD), RMSProp, and Adam. In vehicle control, according to the structure of the split dynamic model, each value function should satisfy the Bellman equation, and under the Bayesian Nash equilibrium condition, the optimal strategy of the two participants can be solved. The optimal strategy for the final operation of the vehicle is determined based on the actual trajectory of the driver and the result of the automation system comprehensively considering two possible combinations.

[0051] S303: Control the vehicle according to the target control combination of the driver and the automation system.

[0052] In this embodiment, a static Bayesian Nash equilibrium shared controller is constructed to improve lane keeping performance. An indirect shared control framework that can integrate the perception, decision-making and control authority of the driver and the automated system is adopted. In this mode, the control input of the smart car is the weighted sum of the driver and the automated system; through the incomplete information controller, a practical incomplete information game hypothesis is performed to derive the optimal strategy expression when the automated system reaches equilibrium under the static Bayesian game theory.

[0053] The dynamics model is updated based on the lane game model of the driver and the automated driving system, wherein the updated dynamics model is used to indicate the strategy function game information of the driver and the automated system under the vehicle dynamics model system with affine input nonlinearity; In this embodiment, firstly, a dynamic model of the vehicle and lane keeping is established; and a lane game model of the driver and the automatic driving system of the vehicle is established; wherein, in the lane game model of the driver and the automatic driving system, the driver and the automatic system have different understandings of the target lane; secondly, based on the pre-established vehicle dynamics model and the lane game model of the driver and the automatic driving system, a Bayesian Nash equilibrium shared control model between the driver and the automatic driving system is obtained; then, the Bayesian Nash equilibrium shared control model is solved to obtain the control conflict information of the driver and the automatic driving system; finally, a shared control strategy is determined according to the control conflict information, so as to control the vehicle according to the shared control strategy. Thus, in the complex game decision-making process in the traffic control system where humans and automatic driving coexist, dynamic game benefits are introduced as feedback, which solves the possible conflict between the individual local optimal strategy and the collective global optimal benefit in group decision-making; the predictive reasoning of game theory and the convergence and stability analysis of control theory are flexibly used to maximize the uncertainty in the learning and decision-making process, so as to promote the establishment of a theoretical framework with stochastic game and stochastic control as the core.

[0054] The present invention focuses on the modeling of nonlinear dynamic systems (affine input nonlinearity) and a fast iterative solution method through approximate dynamic programming. It integrates the driving input of human drivers and automated systems through an intermediate controller, wherein the final control signal of the steering wheel of the car is obtained by weighted summation. Under the framework of non-zero-sum differential game, an affine input nonlinear vehicle dynamics model is established for controller design. Based on the non-zero-sum differential game theory, the driver and the automated system are modeled as two participants. In affine input nonlinear vehicle dynamics, the cooperation and conflict of the participants are easily reflected in their different expected driving trajectories. In view of the nonlinear problem of vehicle dynamics, an asynchronous iterative framework is proposed to solve the Bayesian Nash equilibrium strategy of the automated system, which can obtain the Bayesian Nash equilibrium solution. In order to speed up the iterative solution process of the Nash equilibrium strategy, an offline training neural network is used for approximate dynamic programming, which can be solved online faster than the MPC algorithm (Model Predictive Control).

[0055] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0056] like Fig. 9 , which is a schematic diagram of a game-based collaborative control method device for a human-machine co-driving intelligent vehicle provided by an embodiment of the present invention. A game-based collaborative control method device for a human-machine co-driving intelligent vehicle, the device 900 includes: Modeling module 901 is used to establish a human-machine path control game model corresponding to the human-machine interaction behavior between the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; based on the two expected paths of the driver and the expected path of the automation system, two candidate control combinations of the driver and the automation system are formed; A solution module 902 is used to solve the human-machine path control game model based on the approximate dynamic programming framework model to obtain a target control combination of the driver and the automation system; wherein the target control combination includes target control inputs corresponding to the driver and the automation system respectively; the target control input minimizes the benefit function corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation of the vehicle's driving path from the expected path; The application module 903 is used to control the vehicle according to the target control combination of the driver and the automation system.

[0057] In a preferred embodiment, the modeling module is specifically used to construct a mathematical model of control inputs corresponding to candidate control combinations of the driver and the automation system based on the indirect shared control method between the driver and the automation system; wherein the mathematical model of control inputs is used to indicate the control inputs of the driver and the control inputs of the automation system in the candidate control combinations; based on the control input mathematical model corresponding to the candidate control combinations and a pre-built vehicle dynamics model, construct the respective benefit functions of the driver and the automation system in the candidate control combinations; wherein the vehicle dynamics model is an expression for indicating the vehicle state of the vehicle; based on the respective benefit functions of the driver and the automation system in the two sets of candidate control combinations, a human-machine path control game model is obtained.

[0058] In a preferred embodiment, the modeling module is specifically used to estimate the value of the driver's estimate of the desired path selected by the automated system, which is always 1. The automated system estimates that the probability of the driver selecting one of the two candidate desired paths is , then the probability that the driver chooses the other one is , the vehicle's control input is the control input of the driver and the automation system respectively The weighted sum of , and are all positive numbers not greater than 1. Without considering noise, the control input of the vehicle corresponding to each candidate control combination is expressed as: , In the formula, A weighting factor for the driver's control input; k Represents a time series of samples.

[0059] In a preferred embodiment, the game module is specifically used to construct a two-degree-of-freedom dynamic model of the vehicle based on preset model constraint parameters, wherein the model constraint parameters include: the longitudinal speed of the vehicle is a constant value, the front wheel steering angle of the vehicle satisfies that the tire works in a linear region, and the heading angle deviation satisfies that the distance between the heading of the vehicle and the center line is not greater than a preset value; the two-degree-of-freedom dynamic model of the vehicle is expressed as follows: , In the formula, Represents the vehicle state. In this state expression, represents the two-dimensional ordinate position of the vehicle, represents the lateral velocity of the vehicle, represents the vehicle yaw angle in two-dimensional global coordinates, represents the angular velocity, Indicates the vehicle running speed. Indicates the two-dimensional horizontal coordinate position of the vehicle; represents the observed value; represents the control input, the weight of the turning angle; represents the model error; represents the observation noise; represents the observation matrix; The differential formula of the state variable is as follows, which is constructed without considering the model error. and The specific form is: , In the formula, Indicates the two-dimensional vertical coordinate position of the vehicle The derivative of Represents the vehicle's yaw angle in two-dimensional global coordinates The derivative of Represents the two-dimensional horizontal coordinate position of the vehicle The derivative of Indicates the vehicle running speed The derivative of Indicates the lateral speed of the vehicle The derivative of are the front and rear cornering stiffness, is the vehicle mass, is the polar moment of inertia, is the constant steering ratio, is the longitudinal acceleration, Represent the axial distance from the vehicle's center of mass to the front and rear wheels respectively.

[0060] The vehicle state of the two-degree-of-freedom dynamics model of the vehicle is discretized using the forward Euler method to obtain a vehicle dynamics model with discretized sampling time, including: The vehicle dynamics model with discretized sampling time is expressed as follows: , In the formula, Indicates the time when sampling is performed, Indicates the sampling time point sequence number.

[0061] In a preferred embodiment, the solving module is specifically used to obtain the vehicle state of the vehicle based on the control input of the vehicle corresponding to the candidate control combination and the pre-built vehicle dynamics model. The expression of the vehicle state is as follows: ; In the formula, Indicates the vehicle status. Indicates vehicle status The derivative of A weighting factor for the driver's control input; represents the driver's control input; represents the control input of the automation system; represents the model error; Based on the vehicle state under the control combination, the driver's control strategy for the vehicle and the automation system's control strategy for the vehicle, the corresponding benefit functions of the driver and the automation system in the candidate control combination are constructed; wherein the corresponding benefit functions of the driver and the automation system in the candidate control combination are expressed as: , In the formula, is the corresponding benefit function of the driver in the candidate control combination; is the profit function corresponding to the automation system in the candidate control combination; in, , , In the formula, represents the state of the vehicle in the candidate control combination; is the loss function corresponding to the driver; is the loss function corresponding to the automation system, and They are all second-order positive definite matrices used to adjust the control ratio; The payoff functions of the driver and the automation system satisfy the Bayesian Nash equilibrium of the incomplete information game. , In the formula, represents the control strategy of the automation system, Represents the target control strategy of the automation system; and They represent the driver’s control strategies under two desired paths, p Indicates the inferred probability.

[0062] In a preferred embodiment, the solution module is specifically used to construct an approximate dynamic programming framework model corresponding to each candidate control combination for each candidate control combination; wherein the approximate dynamic programming framework model includes prediction networks corresponding to the driver and the automation system respectively, and the prediction network includes a strategy network and an evaluation network; For each candidate control combination, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system respectively, and the prediction network corresponding to the driver outputs a first current profit function corresponding to the current state of the vehicle and a first next profit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current profit function corresponding to the current state of the vehicle and a second next profit function corresponding to the next state of the vehicle; For each candidate control combination, based on a first next benefit function output by the prediction network corresponding to the driver and a first current loss function of the driver in a current state of the vehicle, a policy network of the prediction network corresponding to the driver is updated, and based on the first next benefit function output by the prediction network corresponding to the driver and a first current benefit function corresponding to the current state of the vehicle, an evaluation network of the prediction network corresponding to the driver is updated; Based on the second benefit function corresponding to the next vehicle state output by each of the two candidate control combinations and the second current loss function of the automation system in the current vehicle state, the evaluation network of the prediction network corresponding to the automation system is updated.

[0063] In a preferred embodiment, the solution module is specifically used to input the current vehicle state of the vehicle into the policy network corresponding to the driver, output a first current control input, and input the current vehicle state into the policy network corresponding to the automation system, output a second current control input; based on the first current control input and the second current control input corresponding to each candidate control combination, predict the next vehicle state of the vehicle, input the predicted next vehicle state into the evaluation network corresponding to the driver, output a first benefit function corresponding to the next vehicle state of the vehicle, and input the predicted next vehicle state into the evaluation network corresponding to the automation system, output a second benefit function corresponding to the next vehicle state of the vehicle.

[0064] Aiming at the incomplete information game problem of human-machine co-driving intelligent vehicles, the present invention firstly establishes a kinematic model of the human-machine co-driving intelligent vehicle, linearizes its nonlinear kinematic model by adopting Taylor expansion on the kinematic model of the vehicle, and then discretizes the linear kinematic model of the vehicle into a discrete linearized model by adopting the forward Euler method; then, the theory of incomplete information game is integrated, the target trajectory benefit of the driver is regarded as an uncertainty factor, and based on the static Bayesian game theory and the Harsanyi transformation, the uncertainty of the driver's benefit is converted into the uncertainty of the road type; secondly, a static Bayesian Nash equilibrium shared controller for improving the lane keeping performance is constructed, and a fusible An indirect shared control framework that combines the perception, decision-making and control authority of the driver and the automated system is proposed. In this mode, the control input of the intelligent car is the weighted sum of the driver and the automated system. Then, an incomplete information controller is designed. Based on the traditional complete information game theory, a practical incomplete information game assumption is made, and the optimal strategy expression when the automated system reaches equilibrium under the static Bayesian game theory is derived; finally, two sets of dual iterative heuristic dynamic programming frameworks are proposed to solve the optimal control strategies of both the driver and the automated driving system under the incomplete information game, and the incomplete information game after the Hasani transformation is transformed into a complete but imperfect information game.

[0065] The above device can execute the game-based collaborative control method for a human-machine co-driving intelligent vehicle provided by an embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the game-based collaborative control method for a human-machine co-driving intelligent vehicle. For technical details not described in detail in this embodiment, please refer to the game-based collaborative control method for a human-machine co-driving intelligent vehicle provided by an embodiment of the present invention.

[0066] The present invention also provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; and a processor for reading executable instructions from the memory and executing the instructions to implement the game collaborative control method of the human-machine co-driving intelligent car of the present invention.

[0067] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0068] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0069] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to the following embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0070] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0071] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0072] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0073] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0074] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0075] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0076] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0077] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0078] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A game-based collaborative control method for a human-machine co-driving intelligent vehicle, characterized in that: The method comprises: Establishing a human-machine path control game model corresponding to the human-machine interaction behavior between the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; forming two candidate control combinations of the driver and the automation system based on the two expected paths of the driver and the expected path of the automation system; Based on the approximate dynamic programming framework model, the human-machine path control game model is solved to obtain the target control combination of the driver and the automation system; wherein the target control combination includes the target control inputs corresponding to the driver and the automation system respectively; the target control inputs minimize the benefit functions corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation of the vehicle's driving path from the expected path; Vehicle control is performed based on a combination of control objectives of the driver and the automation system.

2. The method according to claim 1, characterized in that The establishment of a human-machine path control game model corresponding to the human-machine interaction behavior between the driver and the automation system includes: Based on the indirect shared control mode between the driver and the automation system, a mathematical model of the control input corresponding to the candidate control combination of the driver and the automation system is constructed; wherein the mathematical model of the control input is used to indicate the control input of the driver and the control input of the automation system in the candidate control combination; Based on the control input mathematical model corresponding to the candidate control combination and the pre-constructed vehicle dynamics model, constructing the benefit functions corresponding to the driver and the automation system in the candidate control combination respectively; wherein the vehicle dynamics model is an expression for indicating the vehicle state of the vehicle; The human-machine path control game model is obtained based on the respective corresponding benefit functions of the driver and the automation system in the two groups of candidate control combinations.

3. The method according to claim 2, characterized in that The method of constructing a mathematical model of control input corresponding to the candidate control combination of the driver and the automation system based on the indirect shared control mode between the driver and the automation system includes: The value of the driver's guess about the expected path selected by the automated system is always 1, and the probability that the automated system guesses that the driver will select one of the two candidate expected paths is , then the probability that the driver chooses the other one is , the vehicle's control input is the control input of the driver and the automation system respectively The weighted sum of , and are all positive numbers not greater than 1; without considering noise, the control input of the vehicle corresponding to each candidate control combination is expressed as: ， In the formula, represents the control input of the vehicle, A weighting factor for the driver's control input; k Indicates the sampling time point sequence number.

4. The method according to claim 2, characterized in that: The method further includes constructing a dynamic model of the vehicle, wherein constructing the dynamic model includes: Based on the preset model constraint parameters, a two-degree-of-freedom dynamic model of the vehicle is constructed, wherein the model constraint parameters include: the longitudinal speed of the vehicle is a constant value, the front wheel steering angle of the vehicle satisfies that the tire works in the linear region, and the heading angle deviation satisfies that the distance between the heading of the vehicle and the center line is not greater than a preset value; the two-degree-of-freedom dynamic model of the vehicle is expressed as follows: ， In the formula, Represents the vehicle state. In the vehicle state expression, represents the two-dimensional ordinate position of the vehicle, represents the lateral velocity of the vehicle, represents the vehicle yaw angle in two-dimensional global coordinates, represents the angular velocity, Indicates the vehicle running speed. Indicates the two-dimensional horizontal coordinate position of the vehicle; Indicates the vehicle status Find derivatives; represents the observed value; represents the control input, the weight of the turning angle; represents the model error; represents the observation noise; represents the observation matrix; The differential formula of the state variable is as follows, which is constructed without considering the model error. and The specific form is: ， In the formula, Indicates the two-dimensional vertical coordinate position of the vehicle The derivative of Represents the vehicle's yaw angle in two-dimensional global coordinates The derivative of Represents the two-dimensional horizontal coordinate position of the vehicle The derivative of Indicates the vehicle running speed The derivative of Indicates the lateral speed of the vehicle The derivative of are the front and rear cornering stiffness, is the vehicle mass, is the polar moment of inertia, is the constant steering ratio, is the longitudinal acceleration, Respectively represent the axial distance from the vehicle's center of mass to the front and rear wheels; The vehicle state of the two-degree-of-freedom dynamics model of the vehicle is discretized using the forward Euler method to obtain a vehicle dynamics model with discretized sampling time, including: The vehicle dynamics model with discretized sampling time is expressed as follows: ， In the formula, Indicates the time when sampling is performed, Indicates the sampling time point sequence number.

5. The method according to claim 2, characterized in that: The constructing the benefit functions corresponding to the driver and the automation system in the candidate control combination based on the control input mathematical model corresponding to the candidate control combination and the pre-built vehicle dynamics model includes: Based on the control input of the vehicle corresponding to the candidate control combination and the pre-built vehicle dynamics model, the vehicle state of the vehicle is obtained, and the expression of the vehicle state is as follows: ； In the formula, Indicates the vehicle status. represents the derivative of the vehicle state; represents the driver's control input; represents the control input of the automation system; represents the model error; Based on the vehicle state of the vehicle under the control combination, the control strategy of the driver on the vehicle and the control strategy of the automation system on the vehicle, the benefit functions corresponding to the driver and the automation system in the candidate control combination are constructed; wherein the benefit functions corresponding to the driver and the automation system in the candidate control combination are expressed as: ， In the formula, is the corresponding benefit function of the driver in the candidate control combination; is the profit function corresponding to the automation system in the candidate control combination; in, ， ， In the formula, represents the observed values ​​of the vehicles in the candidate control combination; and represent the target observations of the driver and the automation system respectively; is the loss function corresponding to the driver; is the loss function corresponding to the automation system, and They are all second-order positive definite matrices used to adjust the control ratio; The payoff functions of the driver and the automation system satisfy the Bayesian Nash equilibrium of the incomplete information game. ， In the formula, represents the control strategy of the automation system, Represents the target control strategy of the automation system; and They represent the driver’s control strategies under two desired paths, and They represent the driver’s optimal control strategies under the two desired paths, and represents the driver’s profit function under two expected paths, and represents the profit function of the automation system under two expected paths, p Indicates the inferred probability.

6. The method according to any one of claims 1 to 5, characterized in that: The method of solving the human-machine path control game model based on the approximate dynamic programming framework model to obtain the target control combination of the driver and the automation system includes: For each of the candidate control combinations, construct an approximate dynamic programming framework model corresponding to each of the candidate control combinations; wherein the approximate dynamic programming framework model includes prediction networks corresponding to the driver and the automation system, respectively, and the prediction network includes a strategy network and an evaluation network; For each of the candidate control combinations, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system, respectively, and the prediction network corresponding to the driver outputs a first current profit function corresponding to the current state of the vehicle and a first next profit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current profit function corresponding to the current state of the vehicle and a second next profit function corresponding to the next state of the vehicle; For each of the candidate control combinations, based on a first next benefit function output by the prediction network corresponding to the driver and a first current loss function of the driver in a current state of the vehicle, updating a policy network of the prediction network corresponding to the driver, and based on the first next benefit function output by the prediction network corresponding to the driver and a first current benefit function corresponding to the current state of the vehicle, updating an evaluation network of the prediction network corresponding to the driver; Based on the second benefit function corresponding to the next vehicle state output by each of the two candidate control combinations and the second current loss function of the automation system in the current vehicle state, the evaluation network of the prediction network corresponding to the automation system is updated.

7. The method according to claim 6, characterized in that For each of the candidate control combinations, the current vehicle state of the vehicle is input into the prediction networks corresponding to the driver and the automation system, respectively, the prediction network corresponding to the driver outputs a first current benefit function corresponding to the current state of the vehicle and a first next benefit function corresponding to the next state of the vehicle; the prediction network corresponding to the automation system outputs a second current benefit function corresponding to the current state of the vehicle and a second next benefit function corresponding to the next state of the vehicle, including: Inputting the current vehicle state of the vehicle into a policy network corresponding to the driver and outputting a first current control input, and inputting the current vehicle state into a policy network corresponding to the automation system and outputting a second current control input; Based on the first current control input and the second current control input corresponding to each of the candidate control combinations, the next vehicle state of the vehicle is predicted, the predicted next vehicle state is input into the evaluation network corresponding to the driver, and a first benefit function corresponding to the next vehicle state of the vehicle is output; and the predicted next vehicle state is input into the evaluation network corresponding to the automation system, and a second benefit function corresponding to the next vehicle state of the vehicle is output.

8. A game-based collaborative control device for a human-machine co-driving intelligent vehicle, characterized in that: The device comprises: A modeling module is used to establish a human-machine path control game model corresponding to the human-machine interaction behavior of the driver and the automation system; wherein the human-machine path control game model satisfies the Bayesian Nash equilibrium of the incomplete information game, the driver corresponds to two candidate expected paths, and the automation system corresponds to one expected path; based on the two expected paths of the driver and the expected path of the automation system, two candidate control combinations of the driver and the automation system are formed; A solution module, for solving the human-machine path control game model based on an approximate dynamic programming framework model, and obtaining a target control combination of the driver and the automation system; wherein the target control combination includes target control inputs corresponding to the driver and the automation system respectively; the target control input minimizes the benefit function corresponding to the driver and the automation system respectively; the benefit function is used to indicate the deviation of the vehicle's driving path from the expected path; The application module is used for controlling the vehicle according to the target control combination of the driver and the automation system.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the game-based collaborative control method for the human-machine co-driving intelligent car.

10. A computer-readable medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images