A robot counter-decision-making method based on counterfactual regret minimization

Abstract

This invention relates to a robot adversarial decision-making method based on counterfactual regret minimization, belonging to the field of robot adversarial technology. It includes: acquiring the positional information of both parties; simplifying the decision space using discrete action sets; evaluating the current positions and postures of both parties using a situation evaluation function, and using the situation evaluation function as the basis for decision-making; establishing an imperfect information game model, representing the action choices and positional changes of both robots using a game tree; and searching the game tree model to select a Nash equilibrium as the next motivational action. This invention transforms simultaneous actions into sequences through information set and sequence conversion, considering the simultaneous actions of participants to ensure the algorithm can handle situations involving multiple parties; simultaneously, by introducing Monte Carlo sampling and counterfactual regret minimization algorithms, it effectively improves the efficiency and real-time performance of subgame search.

Description

Technical Field

[0001] This invention belongs to the field of robot adversarial technology and relates to a robot adversarial decision-making method based on minimizing counterfactual regret. Background Technology

[0002] Robot combat plays a role in a wide range of applications, including traffic management, search and rescue operations, and security missions. The combat involves red and blue infantry robots engaging in battle on the battlefield, attempting to engage each other's armored modules. The robots operate within a two-dimensional rectangular space defined by diagonal coordinates. The red and blue robots are identical, sharing the same movement characteristics, including speed and angular velocity. Furthermore, both robots possess matched weapon capabilities. This necessitates a greater focus on the interaction of strategies between the two sides, rather than simply optimizing individual strategies.

[0003] Traditional methods model mobile robot games as differential games in control theory, assuming differential motion models for the participants. A saddle point equilibrium is found by solving the Isaacs equations, which is equivalent to finding a minimum-maximum solution. However, this method assumes complete knowledge of the opponent, which is inconsistent with real-world scenarios.

[0004] Reinforcement learning (RL) methods have attracted considerable attention in solving mobile robot game-playing problems. Various RL methods, such as Q-learning and DQN, have shown promising results. However, simply playing against oneself cannot guarantee convergence to an approximate Nash equilibrium. Summary of the Invention

[0005] The technical problem to be solved by this invention is:

[0006] To overcome the shortcomings of existing technologies and improve decision-making speed and win rate, this invention proposes a counterfactual regret minimization (CFR) algorithm for solving robot adversarial problems. This method combines the ideas of information sets and generalized incomplete information games, effectively addressing the real-time challenges of robot adversarial problems.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A robot adversarial decision-making method based on counterfactual regret minimization, characterized by comprising:

[0009] Obtain the position information of both parties, including position and orientation angle; calculate the distance between the two parties along their line of sight and the angle between their respective headings and line of sight based on the position information;

[0010] The linear velocity and angular velocity of the robot are discretized, and the discrete linear velocity and angular velocity are combined to obtain a discrete motion set;

[0011] The range advantage function is calculated based on the distance between the two sides along the line of sight, and the angle advantage function is calculated based on the angle between their respective headings and the line of sight. The range advantage function, angle advantage function and weapon advantage function are combined to form a comprehensive situation assessment function.

[0012] A game tree is constructed based on the positional information of both parties. The payoff of the leaf nodes is evaluated using a comprehensive situation assessment function. The regret value is calculated based on the payoff and the strategy is updated based on the regret value. The game tree is searched to select maneuver actions from the set of discrete actions.

[0013] A further technical solution of the present invention: the calculation of the distance d along the line of sight between the two parties and the angle φ between their respective headings and the line of sight. r φ b The formula is as follows:

[0014]

[0015]

[0016]

[0017] Where, x r ,y r Indicates the position of the red team's robot, x b ,y b Indicates the position of the blue robot, θ r θ represents the angle between the red robot's orientation and the x-axis. b This indicates the angle between the blue robot's orientation and the x-axis.

[0018] A further technical solution of the present invention: the discrete action set includes 9 different maneuvering actions, where a1 = [0,0], representing stationary; a2 = [0,w], representing stationary; and a1 = [0,w]. r ] indicates a right turn; a3 = [0, w l ] indicates a left turn; a4 = [v f [,0] indicates moving forward; a5 = [v f ,w r ] indicates moving forward to the right; a6 = [v f ,w l ] indicates moving forward to the left; a7 = [v b [,0] indicates back; a8 = [v b ,w r ] indicates moving backward to the right; a9 = [v b ,w l ] indicates moving backward to the left rear; where the angular velocity w = [w l ,0,w r Left turns, maintaining direction, or right turns are permitted; linear velocity v = [vb ,0,v f Controls backward movement, stopping, or moving forward.

[0019] A further technical solution of the present invention: the distance advantage function is calculated based on the distance between the two parties along the line of sight and the angle.

[0020]

[0021] Where D represents the optimal attack distance.

[0022] A further technical solution of the present invention: the calculation of the angle advantage function based on the angle between their respective headings and lines of sight is specifically as follows:

[0023] For the red side, the function is defined as:

[0024]

[0025] For the red side, the function is defined as:

[0026]

[0027] Where, φ r and φ b These represent the angles between the heading and the line of sight for the red and blue robots, respectively.

[0028] A further technical solution of the present invention: the comprehensive situation assessment function is:

[0029] f = f d +f φ +f c

[0030] For the red side, f φ Using the angle advantage function For the blue team, f φ Using the angle advantage function

[0031] A further technical solution of the present invention: the calculation of regret value based on the benefit specifically includes:

[0032] Calculate the virtual payoff v(I) for the subgame information set I in a bottom-up manner:

[0033]

[0034] Where z is a leaf node, The benefit obtained from evaluating the leaf nodes. Let z represent the probability of reaching node z from node I. This represents the probability of reaching node I from the initial state;

[0035] Calculate the regret value r(I) based on the virtual gain v(I) of information set I:

[0036]

[0037] in, This represents the virtual benefit in information set I. Let r(I,a) represent the virtual gain that can be obtained by taking action a on information set I. The difference between r(I,a) and r(A) represents the regret caused by taking action a. If the regret value r(I,a) is greater than 0, it means that taking action a can bring more benefits, and taking action a is a good choice. Conversely, if the regret value r(I,a) is less than 0, it means that taking action a cannot bring more benefits, and taking action a is not a good choice. Therefore, the larger the regret value r(I,a), the better the choice it is to take action a on information set I.

[0038] A further technical solution of the present invention: the update strategy based on regret value specifically includes:

[0039] The strategy is updated at the next iteration t+1 according to the following formula:

[0040]

[0041] in, This indicates taking a positive number, that is, if If it is less than 0, then its value is considered to be 0; if If it is greater than 0, then its value is considered to be the original value; |A(I)| represents the number of elements in the action set A(I) on the information set I, that is, the number of actions that can be taken on the information set I.

[0042] A computer system is characterized by comprising: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0043] The beneficial effects of this invention are as follows:

[0044] This invention provides a robot adversarial decision-making method based on counterfactual regret minimization. Compared with traditional methods and reinforcement learning methods, this invention's method has the following advantages in solving robot adversarial problems:

[0045] 1. Effectively solves the problem of continuous actions and state space: By discretizing actions, the algorithm can cope with complex continuous actions and state space.

[0046] 2. Establish an imperfect information extended game model: By transforming information sets and sequences, simultaneous actions are converted into sequences, taking into account the simultaneous actions of participants, and ensuring that the algorithm can handle situations involving multiple parties.

[0047] 3. Improve search efficiency and real-time performance: By introducing algorithms, Monte Carlo sampling and counterfactual regret minimization are used to effectively improve the efficiency and real-time performance of subgame search.

[0048] Successful verification in practical applications: Through simulation experiments, the effectiveness and success of the algorithm were demonstrated in robot adversarial scenarios. Attached Figure Description

[0049] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0050] Figure 1 A schematic diagram of the effect of discrete action sets.

[0051] Figure 2 Diagram of the confrontation process.

[0052] Figure 3 Game tree diagram.

[0053] Figure 4 A diagram illustrating the subgame search process.

[0054] Figure 5 MCCFRS algorithm flowchart. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0056] This invention provides a robot adversarial decision-making method based on minimizing counterfactual regret, such as... Figure 5 As shown, it includes the following steps:

[0057] 1. Obtain the opponent's position information. The robot combat problem involves two robots, red and blue, with identical performance. During maneuvers, the robots use different combinations of linear and angular velocities to control their movement, aiming to launch weapons and hit the opponent. In this problem, the opponent's control strategy—that is, its angular and linear velocities—is unknown to the other robot.

[0058] 2. Simplify the decision space using a discrete set of actions. Linear velocity and angular velocity are continuous control variables, and their combination creates an excessively large and difficult-to-manage action space. By selecting three definite values ​​each from the continuous linear and angular velocities as the robot's control strategy, a discrete set of actions is obtained.

[0059] 3. A situation assessment function is used to evaluate the current positions and attitudes of both sides, and this function serves as the basis for decision-making. Based on the current positional and attitude relationships between the two sides, three advantage functions are defined from three aspects: distance, angle, and weapon usage. Then, a comprehensive advantage function is proposed based on these three advantage functions. The result of the comprehensive advantage function is used as the current situation assessment result for both sides.

[0060] 4. Establish an imperfect information game model. Represent the action choices and position changes of both robots using a game tree.

[0061] 5. Propose an improved counterfactual regret minimization algorithm for decision-making. Based on the game tree model established in the previous step, perform a game tree search and select the Nash equilibrium as the motivational action for the next step.

[0062] 5.1 Construct a game tree based on the positions of the red and blue teams;

[0063] 5.2. Use the situation assessment function to estimate the value of the results;

[0064] 5.3 Calculate the revenue of the leaf nodes;

[0065] 5.4 Calculate the regret value;

[0066] 5.5 Update node strategy;

[0067] 5.6 Repeat steps 5.2-5.5 until the required number of iterations is reached. Then calculate the average policy and use it as the next maneuver.

[0068] The specific steps described above are as follows:

[0069] 1. Obtain battle status information

[0070] This invention relates to a robot maneuvering strategy method in a combat scenario, rather than robot control technology. Therefore, it focuses only on the two-dimensional projection of the robot and its motion strategy. A red robot (labeled 'r') and a blue robot (labeled 'b') are shown. Each robot's state includes its position (x, y), orientation angle θ, linear velocity v, and angular velocity w. We assume that each robot knows the other's position information (x, y) and orientation angle θ at the current moment, but not the other's maneuvering strategy v, w. When one robot enters the attack range of another robot, it initiates an attack.

[0071] The robot is located in the diagonal coordinates [x min ,y min ] and [x max ,y max Combat takes place within a defined two-dimensional rectangular space. The maximum game duration is... The red and blue robots share the same movement characteristics, including the range of variation in linear and angular velocity. Furthermore, both robots possess matching weapon capabilities, with an optimal attack range of D and an optimal attack angle of φ. max .

[0072] 2. Discretized action space

[0073] This invention first discretizes the robot's linear velocity v and angular velocity ω to reduce the complexity of the problem. Based on the discretized actions, a discrete action set is constructed, containing the different actions the robot can choose. This step allows the robot's behavior to be represented discretely, facilitating the subsequent establishment and solution of the game theory model.

[0074] To study the maneuvering strategies of robots in combat, this invention only considers the robot's kinematic equations.

[0075]

[0076] Here, the state vector (x, y, θ) represents the robot's Cartesian coordinates (x, y) and orientation angle θ in the ground reference frame. The units of linear velocity v and angular velocity ω are m / s and rad / s, respectively.

[0077] Based on the kinematic equation (1), a discrete-time state-space model is used to describe the robot's motion. It is assumed that the control inputs v and ω are piecewise constants within the time interval [kΔt, (k+1)Δt], where Δt>0 is the sampling period. The maximum duration of the entire game process is Δt>0, including the sampling period. The discrete version of the kinematic equations is as follows:

[0078]

[0079] Among them, [x k ,y k ,θ k [v] represents the motion state at step k. k ,ω k ] represents the control input at step k.

[0080] To simplify the problem and enhance the real-time performance of the algorithm, this invention uses a discrete action set to represent the bounded continuous control inputs v and ω. The robot's angular velocity w = [w l ,0,w rLeft turns, maintaining direction, or right turns are permitted. Linear velocity v = [v b ,0,v f Control backward, stop, or move forward. There are 9 different maneuvers to choose from, allowing for diverse strategy combinations. The effects of discrete action sets are as follows: Figure 1 As shown, the corresponding instruction parameters are summarized in Table 1.

[0081] Table 1 Discrete Action Set

[0082]

[0083] 3. Establish a situation assessment function

[0084] To evaluate the effectiveness of the strategy during maneuvering, this invention defines a comprehensive advantage function based on relative position and attack capability. The comprehensive advantage function consists of three parts: a range advantage function, an angle advantage function, and a weapon advantage function.

[0085] The relative geometric positions between the two robots are as follows Figure 2 As shown. The formula is used to calculate the distance d along the line of sight for the red and blue robots, as well as the angle φ between their headings and the line of sight. r φ b .

[0086]

[0087]

[0088]

[0089] Where, x r ,y r Indicates the position of the red team's robot, x b ,y b Indicates the position of the blue robot, θ r θ represents the angle between the red robot's orientation and the x-axis. b This indicates the angle between the blue robot's orientation and the x-axis.

[0090] The distance advantage function is designed to optimize the attack probability while ensuring the robot's safety. It considers the impact of the distance between the two parties on the weapon's attack probability, and is defined as follows:

[0091]

[0092] Where D represents the optimal attack distance. The distance advantage value is within the range (0,1], with a maximum value of 1 at the optimal attack distance.

[0093] The angle advantage function considers the robot's angular position to evaluate the combat situation. For the red side, the function is defined as:

[0094]

[0095] The blue angle advantage function is defined as

[0096]

[0097] Where, φ r and φ b These represent the angles between the red and blue robots' headings and their lines of sight, respectively. The angle advantage varies between 0 and 1, reaching 1 when the red robot is chasing the blue robot and 0 when the blue robot is chasing the red robot.

[0098] In addition, a weapon advantage function is introduced to characterize the feasibility of a successful attack. Based on distance and angle constraints, it examines whether the red robot can hit the target by firing bullets. The function is defined as follows:

[0099]

[0100] Where, φ max Let represent the optimal angle of attack, and e represent a constant.

[0101] Combining the above components, the robot's overall advantage function is obtained:

[0102] f = f d +f φ +f c (10)

[0103] For the Red side, when calculating the overall advantage function, f φ Using the angle advantage function (7); for the blue side, f φ Use the angle advantage function (8).

[0104] Using the comprehensive advantage function as a key indicator to evaluate the robot's advantage in the confrontation process facilitates the evaluation of maneuver strategies.

[0105] 4. Constructing an extended game model with incomplete information

[0106] An imperfect information extended game model is constructed to describe the entire process of the robot confrontation and maneuvering between the two sides.

[0107] In the game, two robots start from an initial state and simultaneously choose a maneuver 'a' from a discrete action set. After a time interval Δt, they move to the next position according to the motion equation (2). They then choose a maneuver again, continuing until the game ends. Here, we present a simplified scenario containing only one time interval. We represent this process using a game tree of depth 2, as follows: Figure 3 As shown. The root node of the game corresponds to the state at the next step. The player acting at the root node is the red robot, which chooses a move from the nine moves in the discrete action set. Then, the blue robot takes its turn. Figure 3 In the diagram, the nine possible nodes of the blue robot are contained within the same dashed area. This means that even though the red robot moves first, the blue robot cannot obtain information about its opponent's actions. In other words, when the red robot moves, the blue robot doesn't know which node it is actually in. This is equivalent to the situation where two robots act simultaneously. Therefore, by choosing an appropriate set of information, a tree structure can also represent simultaneous actions.

[0108] We define an extended game of mobile robot adversarial competition as a tuple It represents the game tree from the initial node to the final result, such as Figure 3 The game tree diagram is shown below.

[0109] • N: The set of participants. In a 1v1 robot battle, we have N = {r, b}, representing the red and blue robots respectively.

[0110] • A: The set of actions. We use a discrete set of actions, denoted as A = {a1, ..., a9}.

[0111] • H: A finite set of game nodes H = {h0, h1, ...}, where each node h represents a node in the game tree. (Symbol) This indicates the parent-child relationship between nodes. For example... Let j be a child node of h. The function p(h) represents the player's decision at that node. Figure 3 In China, we have , p(h0)=r, p(h1)=b.

[0112] • Z: A finite set of leaf nodes Z = {z0, z1, ...}, where each node z represents the terminal history representing the final state of the game. Figure 3 In the middle, leaf node h 10 A terminal node is a node that is reached when the winning conditions are met or when the game time ends in a 1v1 robot match.

[0113] ·u: Utility function Used to evaluate node status. Each player receives a reward u at terminal node z. i (z). In Figure 3 in, u r (h 10 = 1. For a 1v1 robot battle, we define u = 1. r =f r -f b and u b =f b -f r , where f r and f b This is a comprehensive advantage function. It is worth noting that u r =-u b This indicates that the robot adversarial game model is a zero-sum game.

[0114] · Finite set of information set Where I represents the information set. The information set I = {h | p(h) = i} groups nodes together, and players make decisions within this group, without distinguishing specific nodes within the set. In the tree diagram, the information set is represented by dashed lines, which enclose the region containing the relevant nodes. Figure 3 In this case, we have I0 = {h0}, I1 = {h1, ..., h6, ..., h9}

[0115] Player i's strategy σ i Yes Each set of information in Σ is assigned to a function of A. We will then assign Σ to a function of A. i Let σ be the set of all strategies for player i. The strategy set σ consists of the strategies for each player. We use σ... -i This means excluding σ from σ. i The strategy. Definition Let be the probability that all players reach node h if they all choose an action according to strategy σ, where This represents player i's contribution to this probability. For Define π σ (I)=∑ h∈I π σ (h) represents the probability of reaching a specific information set I when all players play according to σ. Finally, define π. σ (h,z)=π σ (z) / π σ (h), It can be defined similarly. and Using this notation, the player's expected reward can be defined as u. i (σ)=∑ h∈Z u i (h)π σ(h).

[0116] The ε-Nash equilibrium is an approximation of the Nash equilibrium and is a set of policies that satisfies the following conditions:

[0117]

[0118] If ε = 0, then the strategy is a Nash equilibrium, meaning no player has any incentive to deviate. Adopting a Nash equilibrium strategy is a good choice in decision-making.

[0119] 5. Subgame Search Decision Method MCCFRS

[0120] This invention proposes a novel Monte Carlo Counterfactual Regret Algorithm (MCCFRS) for depth-finite subgames. It leverages the effectiveness of Monte Carlo sampling in subgames to compute equilibrium values ​​and strategies. This improvement enhances computational efficiency and real-time performance, surpassing traditional CFR methods in subgame search while ensuring equilibrium convergence. MCCFRS is an iterative algorithm for finding the equilibrium value and strategy of a subgame G. At each iteration t, based on the strategy σ... t Sampling is performed on an action 'a' until a terminal node is reached. The specific steps include the following, illustrated in the diagram: Figure 4 As shown.

[0121] 5.1 Constructing the subgame tree G

[0122] Based on the position information of both the red and blue teams, a game tree is constructed. The depth of the game tree is predetermined. The current state of both the red and blue teams, i.e., the red team's position x... r ,y r , towards θ r Blue robot location x b y b , towards θ b Starting with this step, we select different discrete machine actions to extrapolate possible future outcomes and construct a game tree. This step corresponds to... Figure 4 Part (a) of the diagram illustrating the subgame search process.

[0123] During the competition, both robots start from an initial state and simultaneously choose a maneuver 'a' from the discrete action set. After a time interval Δt, they move to the next position according to the motion equation (2). They then choose a maneuver again, continuing until the end of the game, as shown in the example. Figure 3 As shown.

[0124] If constructing a complete game tree requires a lot of resources, we propose an improved method based on Monte Carlo sampling to select one action to be taken in forward deduction, rather than considering all actions in the policy set. That is, we sample only one action and then continue with the subsequent process.

[0125] 5.2 Evaluate the return u(z) of the leaf node.

[0126] The proposed comprehensive situation assessment function is used to evaluate the payoff u(z) of the leaf nodes, where the leaf nodes are the last layer of the game tree, representing the states of both sides in the final step of the deduction. For example, in... Figure 3 In the middle, node h 10 It's just a leaf node. This step corresponds to... Figure 4 Part (b) of the diagram illustrating the subgame search process.

[0127] 5.3 Calculate the virtual revenue v(I)

[0128] The virtual payoff v(I) of the subgame information set I is calculated in a bottom-up manner.

[0129]

[0130] Where z is a leaf node, This refers to the benefit obtained from evaluating the leaf nodes in section 5.2. Let z represent the probability of reaching node z from node I. This represents the probability of reaching node I from the initial state.

[0131] 5.4 Calculate the regret value r(I)

[0132] The regret value r(I) is calculated using the following formula based on the virtual gain v(I) of information set I.

[0133]

[0134] in, This represents the virtual benefit in information set I. Let r(I,a) represent the virtual gain that can be obtained by taking action a on information set I. The difference between r(I,a) and r(A) represents the regret caused by taking action a. If the regret value r(I,a) is greater than 0, it means that taking action a can bring more benefits, and thus taking action a is a good choice. Conversely, if the regret value r(I,a) is less than 0, it means that taking action a cannot bring more benefits, and thus taking action a is not a good choice. Therefore, the larger the regret value r(I,a), the better the choice it is to take action a on information set I.

[0135] By summing up the regrets from multiple iterations on the same information set, we obtain the cumulative regret value R.

[0136]

[0137] Where T represents the number of iterations of the algorithm, the cumulative regret value is obtained by summing the results of multiple iterations in the information set I.

[0138] 5.5 Update Strategy σ

[0139] The strategy is updated according to the following formula at the next iteration t+1.

[0140]

[0141] in, This indicates taking a positive number, that is, if If it is less than 0, then its value is considered to be 0; if If the value is greater than 0, it is considered to be the original value. The goal is to obtain the cumulative positive regret value, because only the action corresponding to the positive regret value is the direction for improvement. |A(I)| represents the number of elements in the action set A(I) on the information set I, that is, the number of actions that can be taken on the information set I.

[0142] The above formula shows that if the cumulative regret value obtained by taking a certain action 'a' on information set I is greater than 0, then the next maneuver is determined according to the proportion. Otherwise, an average strategy is adopted.

[0143] This step corresponds to Figure 4 Part (d) in the diagram of the subgame search process.

[0144] 5.6 Obtaining the average strategy

[0145] The Monte Carlo counterfactual regret algorithm for deep finite subgames converges to a Nash equilibrium as the number of iterations T approaches infinity. By repeatedly running steps 5.2-5.5 above and continuously updating the policy, the resulting average policy is an approximate Nash equilibrium policy, which is the robot's desired maneuver. The formula for calculating the average policy is as follows:

[0146]

[0147] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A robot adversarial decision-making method based on counterfactual regret minimization, characterized in that, include: Obtain the position information of both parties, including position and orientation angle; Calculate the distance between the two sides along their lines of sight, as well as the angle between their respective headings and lines of sight, based on their location information; The linear velocity and angular velocity of the robot are discretized, and the discrete linear velocity and angular velocity are combined to obtain a discrete motion set; The range advantage function is calculated based on the distance between the two sides along the line of sight, and the angle advantage function is calculated based on the angle between their respective headings and the line of sight. The range advantage function, angle advantage function and weapon advantage function are combined to form a comprehensive situation assessment function. The distance advantage function is calculated based on the distance between the two parties along their line of sight: in, Indicates the optimal attack range. The distance between the two parties along their line of sight; The calculation of the angle advantage function based on the angle between their respective headings and lines of sight is specifically as follows: For the red side, the function is defined as: For the blue team, the function is defined as: in, and These represent the angles between the heading and the line of sight of the red and blue robots, respectively. The weapon advantage function is defined as: in, Indicates the optimal angle of attack. Represent a constant; Combining the above components, the robot's overall advantage function is obtained: A game tree is constructed based on the positional information of both parties. The payoff of the leaf nodes is evaluated using a comprehensive situation assessment function. The regret value is calculated based on the payoff, and the strategy is updated based on the regret value. The above process is iterated to obtain the average strategy. Maneuver actions are selected from the discrete action set based on the equilibrium strategy.

2. The robot adversarial decision-making method based on counterfactual regret minimization according to claim 1, characterized in that, The calculation of the distance between the two parties along their line of sight and the angle between their respective headings and lines of sight. , The formula is as follows: in, Indicates the position of the red team's robot. Indicates the location of the blue team's robot. Indicates the direction of the red team's robot towards... x The included angle of the axis, Indicates the orientation of the blue robot towards... x The included angle of the axis.

3. The robot adversarial decision-making method based on counterfactual regret minimization according to claim 1, characterized in that, The discrete action set includes nine different maneuvers, among which = , indicating no movement; = , indicates a right turn; = , indicates a left turn; = , indicating forward movement; = This indicates advancing to the right front; = This indicates moving forward to the left front; = , indicates moving backward; = , indicating retreating to the right rear; = , indicating backward movement to the left rear; where angular velocity Left turns, maintaining direction, or performing right turns are permitted; linear speed Control backward movement, stopping, or moving forward.

4. The robot adversarial decision-making method based on counterfactual regret minimization according to claim 1, characterized in that, The specific steps for calculating the regret value based on the returns are as follows: Calculate the subgame information set in a bottom-up manner. Virtual income : in, It is a leaf node. The benefit obtained from evaluating the leaf nodes. Indicates from node Reaching the node The probability, This indicates the arrival at a node from the initial state. The probability of; According to the information set Virtual income Calculate regret value : in, Indicates in the information set Virtual income, Indicates in the information set Take action The virtual gains obtained, therefore, regret value The larger the value, the more it indicates the information set. Take action It's a better choice.

5. A robot adversarial decision-making method based on counterfactual regret minimization according to claim 4, characterized in that, The update strategy based on regret value is specifically as follows: According to the following formula, in the next iteration Update strategy: in, This indicates that the regret value is positive, i.e., if If it is less than 0, then its value is considered to be 0; if If the value is greater than 0, then its value is considered to be the original value; Indicates in the information set Up Action Set The number of elements in the information set, that is, the number of elements in the information set. The number of actions that can be taken.

6. A robot adversarial decision-making method based on counterfactual regret minimization according to claim 5, characterized in that, The formula for calculating the average strategy is as follows: 。 7. A computer system, characterized in that... include: One or more processors, a computer-readable storage medium, for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method of any one of claims 1-6.

Images