Probability reachable set calculation method based on multi-agent random pursuit game

Abstract

The invention relates to a probability reachable set calculation method based on a multi-agent random pursuit game, and belongs to the field of multi-agent pursuit games. Comprising the following steps: respectively establishing kinetic equations of an attacker and a defender according to the speeds of the attacker and the defender; constructing a value function based on the kinetic equations of the attacker and the defender; and carrying out Taylor expansion on the value function by adopting the Ittan integral to obtain a probability reachable set result. According to the method, a current pursuit game solving method is expanded to a more general framework, and a random part of system dynamics is considered; establishing a relation between a random pursuit game problem and a random optimal control problem related to a discontinuous revenue function, and considering interaction among a plurality of agents in the game; and the problem of multi-agent random pursuit game is solved by utilizing probability reachability analysis.

Description

Technical Field

[0001] The present invention belongs to the field of multi-agent pursuit and escape games, and specifically relates to a method for calculating a probabilistic reachable set based on a multi-agent random pursuit and escape game. Background Art

[0002] Differential game theory has been widely used in robotics, spacecraft, aircraft control, and other fields. In differential games, the pursuit-escape game is defined between two opposing teams of cooperating agents, where the goal of one team is to reach a certain goal and the goal of the other team is to prevent their opponent from succeeding. This game is an excellent proxy tool for studying the tools needed to achieve intelligent automated systems.

[0003] Several aspects of multiplayer pursuit games have been explored. Many previous studies have focused more on the multi-agent cooperative aspects of the game, while making simplifying assumptions about the adversarial aspects. In more complex pursuit games, the roles of the agents may change over time. The opponent's actions are predicted based on the expected opponent's strategy, and feedback and replanning are used to adjust for deviations from the predicted actions at runtime. In the more general case, where neither side has known prior information, an ideal framework is the Hamilton-Jacobi-Isaacs reachability method, where the Hamilton-Jacobi-Isaacs partial differential equations are solved to obtain the optimal strategies for both teams.

[0004] Most pursuit-escape games are studied in a deterministic setting, ignoring the uncertain effects of system dynamics and the environment. Therefore, it is necessary to extend the current pursuit-escape game solution to a more general framework, in which the stochastic part of the system dynamics should be considered. Summary of the invention

[0005] The technical problems to be solved by the present invention are:

[0006] In order to avoid the shortcomings of the prior art, the present invention provides a probabilistic reachable set calculation method based on multi-agent random pursuit and escape game, which is used to solve the uncertainty impact of system dynamics and environment. From the following two aspects:

[0007] (1) Establish the connection between the stochastic pursuit-escape game problem and the stochastic optimal control problem involving discontinuous payoff functions, and consider the interaction between multiple agents in the game.

[0008] (2) Use probabilistic reachability analysis to solve the problem of multi-agent random pursuit and escape game.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0010] A method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game, wherein the multi-agent is divided into an attacker and a defender, and includes:

[0011] The dynamic equations of the attacker and defender are established according to their speeds respectively;

[0012] Construct a value function based on the dynamic equations of the attacker and defender;

[0013] The value function is Taylor expanded using Ito integral to obtain the result of the probabilistic reachable set.

[0014] A further technical solution of the present invention is as follows: the kinetic equations of the attacker and the defender are established according to their speeds, specifically:

[0015]

[0016]

[0017] Each attacker has the same maximum speed v A , each defender has the same maximum speed v D , a i (·),d j (·) represent the control functions of the attacker and defender, respectively. represents the speed of the power system of the deterministic part of the attacker and defender; denote the disturbance terms of the Brownian motion of the attacker and the defender, respectively, are the initial states of the attacker and defender at time 0 respectively.

[0018] A further technical solution of the present invention is as follows: the value function is constructed based on the kinetic equations of the attacker and the defender, specifically:

[0019] The optimal control framework is combined with the time evolution of the probability density function to generate a probabilistic reachability analysis domain. The probabilistic reachability analysis will calculate the transition probability to a certain state; the value function will be equivalent to being in the initial set at time t = 0 and the conditional probability of being in the final state x at time t = T;

[0020] Consider a controlled random process

[0021]

[0022] in, Joint control input for the attacking team, Combined control input for the defending team; assembly and denote the joint admissible control input set of the attacking team and the defending team respectively; f(x,u,d) is the dynamic system function of the deterministic part of the attacker and the defender

[0023]

[0024]

[0025]

[0026] Because the dynamics are stochastic, the value function cannot be minimized; instead, the expected reward is minimized over all possible further realized Wiener processes

[0027]

[0028] set up is a non-empty target set. express expectations;

[0029] Consider the reachable set Ω with success probability ρ∈[0,1] t , or the trajectory where the initial condition x exists Arrives at the set at time T The probability set

[0030]

[0031] Among them, the collection It can be expressed by using the level set method

[0032]

[0033] in,

[0034] If we assume that the value function is differentiable, we can derive:

[0035]

[0036] in, is a zero-mean process with a normal distribution and a standard deviation that is a function of state and time.

[0037] A further technical solution of the present invention is: the Ito integral is used to perform Taylor expansion on the value function to obtain a probability reachable set result, which is specifically:

[0038] Due to the Wiener process, dx 2 is of order dt, so the expansion must be performed to second order:

[0039]

[0040] Where Δx=Δf(x,u,d)+ξ. Keep all order terms Using the Dynkin operator:

[0041]

[0042] Substitute into V(x,t) and divide by Δt

[0043]

[0044] For the limit Δt→0, the value function is given by the viscosity solution of the stochastic Hamilton–Jacobi–Bellman equation

[0045]

[0046] The stochastic Hamilton–Jacobi–Bellman equation can be solved in a similar way to the deterministic Hamilton–Jacobi–Bellman equation; in contrast to deterministic reachable sets, the value function in probabilistic reachability analysis is not expressed as a signed distance function; instead, it is expressed using an indicator function:

[0047]

[0048] The regularization of the indicator function is expressed as:

[0049]

[0050] Among them, V ε (x) is The Lipschitz continuum.

[0051] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0052] A computer-readable storage medium, characterized in that it stores computer-executable instructions, which are used to implement the above method when executed.

[0053] A computer program product, characterized by comprising computer executable instructions, wherein the instructions are used to implement the above method when executed.

[0054] The beneficial effects of the present invention are:

[0055] The present invention provides a method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game, which can be applied to reachability analysis and a multi-agent pursuit-escape game to calculate a probabilistic reachable set of a multi-agent random pursuit-escape game.

[0056] (1) Most pursuit-escape games are studied in a deterministic environment, ignoring the uncertainty of system dynamics and the environment. This invention extends the current pursuit-escape game solution to a more general framework and considers the stochastic part of the system dynamics.

[0057] (2) Establish the connection between the random pursuit-escape game problem and the stochastic optimal control problem involving discontinuous payoff functions, and consider the interaction between multiple agents in the game.

[0058] (3) Use probabilistic reachability analysis to solve the problem of multi-agent random pursuit and escape game. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0060] Figure 1 It is a schematic diagram of a multi-agent pursuit-escape game of the present invention;

[0061] Figure 2 is the calculation result of the probabilistic reachable set of the attacker relative to each defender of the present invention;

[0062] Figure 3 It is the calculation result of the probabilistic reachable set of the defender of the present invention relative to each attacker.

[0063] Figure 4 It is a flow chart of the method of the present invention.

[0064] Figure 5 It is a framework diagram of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0066] The present invention proposes a method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game. First, the winning conditions for the attacker to reach the target and the winning conditions for the defender to prevent the attacker from reaching the target are established. Next, the optimal control framework is combined with the time evolution of the probability density function to generate a probabilistic reachability analysis domain. The probabilistic reachability analysis calculates the transition probability to a certain state. Then, the stochastic Hamilton-Jacobi-Bellman equation is solved according to the deterministic Hamilton-Jacobi-Bellman equation, and the probabilistic reachable set result is obtained.

[0067] like Figure 2 As shown, the specific steps include:

[0068] Step 1: The present invention first establishes the winning conditions for the attacker to reach the target and the winning conditions for the defender to prevent the attacker from reaching the target.

[0069] Establish a multi-agent pursuit-escape game between N attackers and N defenders, such as Figure 1 Each agent is confined to a bounded open domain It can be divided into a set of obstacle domains Ω obs and free space Ω free .

[0070] set up denote the states of the attacker and defender respectively. The initial conditions of the agent are In the random pursuit-escape game, the system dynamics consists of a deterministic part and a stochastic part. The deterministic part of the dynamics is represented by the following decoupled system:

[0071]

[0072]

[0073] Among them, a i (·),d j (·) represent the control functions of the attacker and defender respectively. Each attacker has the same maximum speed v A , each defender has the same maximum speed v D . Represents the dynamical system (velocity) of the deterministic parts of the attacker and defender.

[0074] Assuming that the movements of both the attacker and defender are subject to Brownian motion W, the dynamics of the agent can be expressed by the following stochastic differential equation:

[0075]

[0076]

[0077] in, represents the disturbance term of the Brownian motion.

[0078] The joint state and joint initial conditions of the agents become

[0079] In this pursuit-escape game, the attacker's goal is to reach Figure 1 The green targets (a tight subset) are shown without being captured by the defender.

[0080] Capture conditions are represented as capture sets consisting of pairs of agents The present invention defines the capture set as

[0081]

[0082] Among them, if the relative distance between the attacker and the defender is R C If the attacker is captured by the defender.

[0083] Step 2: Combine the optimal control framework with the time evolution of the probability density function to generate the probabilistic reachability analysis domain. The probabilistic reachability analysis will calculate the transition probability to a certain state. In other words, the value function will be equivalent to being in the initial set at t = 0 and the conditional probability of being in the final state x at time t=T.

[0084] Consider a controlled random process

[0085]

[0086] in, Joint control input for the attacking team, Combined control input for the defending team. and denote the joint admissible control input set of the attacking team and the defending team respectively. f(x,u,d) is the dynamic system function of the deterministic part of the attacker and the defender

[0087]

[0088]

[0089]

[0090] Because the dynamics are stochastic, the value function cannot be minimized. Instead, the expected reward is minimized over all possible further realized Wiener processes

[0091]

[0092] set up is a non-empty target set. Express expectations.

[0093] Consider the reachable set Ω with success probability ρ∈[0,1] t , or the trajectory where the initial condition x exists Arrives at the set at time T The probability set

[0094]

[0095] Among them, the collection It can be expressed by using the level set method

[0096]

[0097] in,

[0098] If we assume that the value function is differentiable, we can derive:

[0099]

[0100] in, is a zero-mean process with a normal distribution and a standard deviation that is a function of state and time.

[0101] Step 3: Use Ito integral to perform Taylor expansion on the value function. Due to the Wiener process, dx 2 is of order dt, so the expansion must be performed to second order

[0102]

[0103] Where Δx=Δf(x,u,d)+ξ. Keep all order terms Using the Dynkin operator:

[0104]

[0105] Substitute into V(x,t) and divide by Δt

[0106]

[0107] For the limit Δt→0, the value function is given by the viscosity solution of the stochastic Hamilton–Jacobi–Bellman equation

[0108]

[0109] The stochastic Hamilton-Jacobi-Bellman equation can be solved in a similar way to the deterministic Hamilton-Jacobi-Bellman equation. In contrast to deterministic reachable sets, the value function in probabilistic reachability analysis is not represented by a signed distance function. Instead, it is represented by an indicator function:

[0110]

[0111] The regularization of the indicator function is expressed as:

[0112]

[0113] Among them, V ε (x) is The Lipschitz continuum.

[0114] exist Figure 3 In each subgraph of , the probability of the attacker "winning" area relative to each defender is shown (ρ=0.8). The probability of an attacker initially located in this area winning is ρ=0.8. For example, in the upper left subgraph, the probability of losing to all four attackers is 0.8 because the defender is too far from the goal, while in the lower left subgraph, the probability of the defender winning against three of the four attackers is 0.8 because it is close to the goal. Therefore, Figure 3 The shape of the probabilistically reachable set in is determined by the initial positions of the four defenders. Similarly, Figure 4 The defender's probabilistic "win" region is shown, where the defender is guaranteed to win against the attacker with a high probability of 0.8. For example, Figure 4 As shown in the upper right figure, due to the short distance to the target, only one of the four defenders has a higher probability of winning against the attacker.

[0115] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game, characterized in that: The multi-agent is divided into attackers and defenders, including: The dynamic equations of the attacker and defender are established according to their speeds respectively; Construct a value function based on the dynamic equations of the attacker and defender; The value function is Taylor expanded using Ito integral to obtain the result of the probabilistic reachable set.

2. According to claim 1, a method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game is characterized in that: The dynamic equations of the attacker and the defender are established according to their speeds, specifically: Each attacker has the same maximum speed v A , each defender has the same maximum speed v D , a i (·),d j (·) represent the control functions of the attacker and defender, respectively. represents the speed of the power system of the deterministic part of the attacker and defender; denote the disturbance terms of the Brownian motion of the attacker and the defender, respectively, are the initial states of the attacker and defender at time 0 respectively.

3. According to claim 2, a method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game, It is characterized in that The value function is constructed based on the dynamic equations of the attacker and the defender, specifically: The optimal control framework is combined with the time evolution of the probability density function to generate a probabilistic reachability analysis domain, which will calculate the transition probability to a certain state; The value function will be equivalent to being in the initial set at time t = 0 and the conditional probability of being in the final state x at time t = T; Consider a controlled random process in, Joint control input for the attacking team, Combined control input for the defending team; assembly and denote the joint admissible control input set of the attacking team and the defending team respectively; f(x,u,d) is the dynamic system function of the deterministic part of the attacker and the defender Because the dynamics are stochastic, the value function cannot be minimized; instead, the expected reward is minimized over all possible further realized Wiener processes set up is a non-empty target set. express expectations; Consider the reachable set Ω with success probability ρ∈[0,1] t , or the trajectory where the initial condition x exists Arrives at the set at time T The probability set Among them, the collection It can be expressed by using the level set method in, If we assume that the value function is differentiable, we can derive: in, is a zero-mean process with a normal distribution and a standard deviation that is a function of state and time.

4. The method for calculating a probabilistic reachable set based on a multi-agent random pursuit-escape game according to claim 3, characterized in that: The Ito integral is used to perform Taylor expansion on the value function to obtain the probability reachable set result, which is specifically: Due to the Wiener process, dx 2 is of order dt, so the expansion must be performed to second order: Where Δx=Δf(x,u,d)+ξ. Keep all order terms Using the Dynkin operator: Substitute into V(x,t) and divide by Δt For the limit Δt→0, the value function is given by the viscosity solution of the stochastic Hamilton–Jacobi–Bellman equation The stochastic Hamilton–Jacobi–Bellman equation can be solved in a similar way to the deterministic Hamilton–Jacobi–Bellman equation; in contrast to deterministic reachable sets, the value function in probabilistic reachability analysis is not expressed as a signed distance function; instead, it is expressed using an indicator function: The regularization of the indicator function is expressed as: Among them, V ε (x) is The Lipschitz continuum.

5. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

6. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

7. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Images