Method, device and equipment for generating pursuit strategy of multi-robot pursuit game

Abstract

The invention relates to the technical field of pursuit game, and discloses a pursuit strategy generation method, device and equipment for a multi-robot pursuit game, and the method comprises the steps: calculating the pursuit strategies of all pursuit robots and the escape strategies of escape robots in a training map based on the type of the training map; training the initial strategy model based on the chasing strategies and the escape strategies in all the training maps to obtain a target strategy model; and for the target map, adopting the target strategy model to generate target chasing strategies of all the chasing robots in the target map. According to the method, strategies of pursuing and escaping parties are respectively generated aiming at different types of training maps, the model learns strategy characteristics under different map structures based on the training model, the limitation that the map structure is single and the escape strategy is simplified is broken through, the method has zero sample generalization ability, the trained model is applied to an unknown target map, and the method has high practical value. An adaptive pursuit strategy can be quickly generated, and actual combat requirements in a complex environment are met.

Description

Technical Field

[0001] This invention relates to the field of pursuit and escape game technology, specifically to a method, apparatus, and equipment for generating pursuit strategies in multi-robot pursuit and escape games. Background Technology

[0002] In emergency scenarios within the public safety field, multi-agent pursuit-escape game theory has become an important research direction. This involves deploying pursuing and escaping robots to simulate real-world containment and interception tasks. The escaping robots attempt to avoid the pursuing robots and reach unoccupied exits, while the pursuing robots cooperate to prevent the escaping robots from reaching the exits earlier or directly surrounding them. Therefore, how to generate pursuit strategies for the pursuing robots has become a crucial issue in pursuit-escape game theory.

[0003] Existing pursuit strategy generation methods are mostly designed for fixed, single map structures and oversimplify the escape strategies of escaping robots, such as assuming fixed escape paths and lacking dynamic adversarial awareness. This directly leads to the fact that even pursuit strategies optimized through deep learning are difficult to break through the constraints of the map structure. In other words, they cannot generalize in completely new and unknown maps, and because they do not consider the adaptive adversarial capabilities of escaping robots, they are easily breached by the pursuing robots, making it difficult to meet the practical needs of complex public safety scenarios. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus and equipment for generating pursuit strategies in a multi-robot pursuit game, in order to solve the problems that existing pursuit strategy generation methods cannot be generalized in new and unknown maps, and are easily breached by pursuing robots because they do not consider the adaptive and adversarial capabilities of escaping robots, making it difficult to meet the practical needs of complex public safety scenarios.

[0005] In a first aspect, the present invention provides a method for generating a pursuit strategy in a multi-robot pursuit game, the method comprising:

[0006] For each training map, based on the type of the training map, calculate the pursuit strategy of all pursuing robots and the escape strategy of all escaping robots in the training map.

[0007] Based on the pursuit and escape strategies in all training maps, train the initial strategy model to obtain the target strategy model;

[0008] For the target map, a target strategy model is used to generate target pursuit strategies for all pursuit robots in the target map.

[0009] This invention generates strategies for both the pursuer and the pursuer using adapted algorithms for different types of training maps. By training the model with all training maps and their pursuit strategies, the model learns the strategy features under different map structures, overcoming the limitations of simple map structures and escape strategies. It has zero-shot generalization ability and can directly apply the trained target strategy model to unknown target maps to quickly generate adapted pursuit strategies. These strategies are not easily exploited by adaptive opponents, improving the pursuit success rate in complex game scenarios and meeting the practical needs of complex environments.

[0010] In one alternative implementation, the training map is of type no exit;

[0011] For each training map, based on the type of the training map, calculate the pursuit strategy of all pursuing robots and the escape strategy of all escaping robots in the training map, including:

[0012] All possible discrete positions of all pursuing robots in the training map are taken as pursuing nodes, and all possible discrete positions of escaped robots are taken as escape nodes. Edges are constructed based on the positional relationships between all pursuing nodes and escape nodes to form the graph structure corresponding to the training map.

[0013] Initialize the queue;

[0014] For each pursuit node, determine whether the positions of the pursuit node and the escape node satisfy the game function;

[0015] When the positions of the pursuing node and the escape node satisfy the game function, add the positions of the pursuing node and the escape node as a set of game states to the queue, and set the number of pursuit steps for the game state.

[0016] When the queue is not empty, retrieve the game state at the head of the queue and determine the game states of its neighbors.

[0017] The number of pursuit steps for each neighboring game state is updated based on the game state, and the neighboring game states are added to the queue. The process is repeated until the queue is empty, at which point the first game state in the queue is retrieved and its neighboring game states are determined. This process is repeated until the queue is empty, and the number of pursuit steps for all game states in the graph structure is obtained.

[0018] Graph-based computation of pursuit and escape strategies.

[0019] This invention abstracts the pursuing and escaping robots as nodes, constructs edges based on positional relationships, and transforms the physical map into a graph theory model. This allows complex spatial relationships to be analyzed using graph algorithms. Effective game states are selected through a game function, and the pursuit steps are initialized for game states that conform to the game function. By determining the game states of neighbors, the most likely next action is simulated. The pursuit steps for all game states are updated to obtain the total pursuit steps for all game states in the entire graph. The pursuit strategy is calculated using these pursuit steps, conforming to Nash equilibrium logic. This ensures that the strategies of both sides reach their optimal state in mutual confrontation and considers the adaptive countermeasure capabilities of the escaping robot, thus improving the reliability of the strategy.

[0020] In one alternative implementation, the pursuit strategy and escape strategy are calculated based on the graph structure, including:

[0021] Identify multiple first neighbor nodes of each pursuing node and multiple second neighbor nodes of each escape node;

[0022] For each first neighbor node, determine the number of pursuit steps for the game state formed by each second neighbor node and the first neighbor node, and take the maximum value among all pursuit steps as the first target step number corresponding to the first neighbor node.

[0023] The minimum value among all the first target steps is taken as the second target step, and the first neighbor node corresponding to the second target step is taken as the pursuit strategy of the pursuit robot corresponding to the pursuit node.

[0024] For each second neighbor node, determine the number of pursuit steps for the game state formed by each first neighbor node and second neighbor node, and take the minimum value among all pursuit steps as the third target step number corresponding to the second neighbor node.

[0025] The maximum value among all the third target steps is taken as the fourth target step, and the second neighbor node corresponding to the fourth target step is taken as the escape strategy of the escape robot corresponding to the escape node.

[0026] This invention generates pursuit and escape strategies through extreme value optimization logic. Essentially, it is a pure strategy approximation of Nash equilibrium between the pursuer and the escapee in a graph structure, so that the strategies of the pursuer and the escapee are mutually constrained. Neither side can gain an advantage by changing its actions alone, thus ensuring that the strategy remains stable in dynamic games.

[0027] In one alternative implementation, the training map is of the type with an exit;

[0028] For each training map, based on the type of the training map, the pursuit strategy of all pursuing robots and the escape strategy of all escaping robots in the training map are calculated, including:

[0029] Construct an initial bipartite graph corresponding to the training map, remove nodes without connected edges from the initial bipartite graph, and obtain the target bipartite graph;

[0030] For each exit node in the target bipartite graph, determine the first shortest distance between each exit node and the escape robot, and sort all the first shortest distances in ascending order;

[0031] Determine the maximum number of defensible exits. From all sorted first shortest distances, determine the maximum number of defensible first shortest distances and the corresponding target exit nodes.

[0032] For each target exit node, identify target pursuit nodes that have a matching relationship with the target exit node from the target bipartite graph;

[0033] For each target pursuit node, the pursuit strategy is to go to the exit node that has a matching relationship with the target pursuit node.

[0034] For each other pursuit node besides the target pursuit node, the pursuit strategy will be to go to the corresponding exit node based on the first shortest distance after sorting the edges of the target pursuit node.

[0035] For escape robots, the escape strategy is to prioritize going to the removed exit node. If there is no removed exit node, the robot will go to the corresponding exit node according to the first shortest distance after sorting.

[0036] This invention, by using bipartite graph matching and priority ranking for scenarios with exits, achieves optimized allocation of pursuit resources and targeted countermeasures against escape paths. The generated strategy combinations approach Nash equilibrium, providing high-quality training samples for subsequent reinforcement learning and helping the target policy model learn the patterns in complex exit scenarios.

[0037] In one optional implementation, constructing an initial bipartite graph corresponding to the training map includes:

[0038] Each exit in the training map is taken as an exit node, and each pursuit robot is taken as a pursuit node.

[0039] For each pursuit node and each exit node, determine the second shortest distance from the pursuit node to the exit node, and determine the third shortest distance from the escape robot to the exit node.

[0040] When the second shortest distance is no greater than the third shortest distance, construct an edge between the pursuit node and the exit node;

[0041] Construct an initial bipartite graph based on all exit nodes, all pursuit nodes, and all edges.

[0042] This invention constructs an initial bipartite graph, and the node and edge structure demonstrates the matching possibility between the pursuer and the exit, providing a clear topological basis for subsequent steps and improving the scientific nature of the strategy.

[0043] In one optional implementation, an initial policy model is trained based on the pursuit and escape policies in all training maps to obtain a target policy model, including:

[0044] Randomly select any training map from all training maps, input the training map into the initial policy model, and generate a joint policy;

[0045] The current position of the escape robot and the current position of each pursuing robot are randomly selected, and the current action of each pursuing robot is selected based on a joint strategy.

[0046] For each pursuing robot's current position, based on the pursuing robot's pursuit strategy in the training map, sample actions of the pursuing robot are generated, and based on the escape robot's escape strategy in the training map, game actions of the escape robot are generated.

[0047] Based on the current position and sampled actions of the pursuing robot, as well as the current position and game actions of the escaping robot, a reward signal, the next position of the pursuing robot, and the next position of the escaping robot are generated.

[0048] The current position, next position, and current action of the pursuit robot, the current position, next position, game action of the escape robot, and the reward signal are used as a set of training samples;

[0049] Return to the step of randomly selecting any training map from all training maps, inputting the training map into the initial policy model, and generating a joint policy until the number of training samples reaches a preset threshold, and calculate the training loss based on all training samples and the joint policy;

[0050] Based on the training loss, update the model parameters of the initial policy model, return to the step of randomly selecting any training map from all training maps, inputting the training map into the initial policy model, and generating a joint policy, until the model parameters converge to obtain the target policy model.

[0051] This invention enables the model to overcome the limitations of a single map through iterative learning on diverse training maps. It can be directly applied to target maps with unknown structures, achieving zero-sample transfer. Strongly adversarial game samples and theoretically optimal sampling actions jointly guide the model, making its generated strategies close to the theoretical optimum and able to cope with adaptive escape behavior, making them difficult for opponents to exploit. The design and iterative optimization of joint strategies ensure that the model can learn the division of labor and cooperation logic of multiple robots, improving the success rate of pursuit in complex scenarios.

[0052] In one alternative implementation, the initial policy model includes a feature extractor, an encoder, a decoder, and a pointer network;

[0053] The training map is input into the initial policy model to generate a joint policy, including:

[0054] The training map is input into the feature extractor in the initial policy model to generate the state features of each pursuit robot.

[0055] The state features are input into the encoder to obtain local features;

[0056] Local features are input into the decoder to obtain global features;

[0057] The local and global features are concatenated and then input into the pointer network to obtain the training and pursuit strategy for each pursuit robot, thus obtaining the joint strategy.

[0058] This invention combines a feature extractor, encoder, decoder, and pointer network to preserve local topological details while integrating global situational awareness. This aligns state feature inputs with different graph structures, and policy outputs further adapt to the topological differences of different maps. Sequential iteration ensures that subsequent decisions depend on preceding actions, and the product form of the joint policy strengthens the implicit cooperation of multiple agents, avoiding global inefficiency caused by individual optimality. This lays the foundation for subsequent zero-sample migration to unknown maps.

[0059] Secondly, the present invention provides a pursuit strategy generation device for multi-robot pursuit and escape game, the device comprising:

[0060] The calculation module is used to calculate the pursuit strategy of all pursuing robots and the escape strategy of escaping robots for each training map, based on the type of the training map.

[0061] The training module is used to train the initial policy model based on the pursuit and escape strategies in all training maps, and obtain the target policy model.

[0062] The generation module is used to generate target pursuit strategies for all pursuit robots in the target map using a target strategy model.

[0063] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the pursuit strategy generation method of the multi-robot pursuit game described in the first aspect or any corresponding embodiment.

[0064] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the pursuit strategy generation method for a multi-robot pursuit game described in the first aspect or any of its corresponding embodiments. Attached Figure Description

[0065] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0066] Figure 1 This is a flowchart illustrating a method for generating a pursuit strategy in a multi-robot pursuit game according to an embodiment of the present invention.

[0067] Figure 2 This is a flowchart of generating a pursuit strategy in an exit training map according to an embodiment of the present invention;

[0068] Figure 3 This is a flowchart of a strategy model according to an embodiment of the present invention;

[0069] Figure 4 This is a structural block diagram of a pursuit strategy generation device for a multi-robot pursuit game according to an embodiment of the present invention;

[0070] Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Existing methods for generating pursuit strategies are mostly designed for fixed, single map structures and oversimplify the escape strategies of escaping robots. This results in a failure to generalize to entirely new, unknown maps and, due to a lack of consideration for the adaptive adversarial capabilities of escaping robots, makes them vulnerable to being breached by pursuing robots, failing to meet the practical needs of complex public safety scenarios. This invention generates strategies for both the pursuer and the escaper using adapted algorithms for different types of training maps. By training the model with all training maps and their pursuit strategies, the model learns the strategy characteristics under different map structures, overcoming the limitations of single map structures and simplified escape strategies. It possesses zero-shot generalization capability, allowing the trained model to be directly applied to unknown target maps, quickly generating adapted pursuit strategies that are less susceptible to exploitation by adaptive opponents. This improves the success rate of pursuit in complex game scenarios and meets the practical needs of complex environments.

[0073] According to an embodiment of the present invention, a method for generating a pursuit strategy in a multi-robot pursuit game is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0074] This embodiment provides a method for generating pursuit strategies in a multi-robot chase game. Figure 1 This is a flowchart of a method for generating a pursuit strategy in a multi-robot pursuit game according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0075] Step S101: For each training map, based on the type of the training map, calculate the pursuit strategy of all pursuing robots and the escape strategy of all escaping robots in the training map.

[0076] Specifically, in real-world public safety scenarios, the pursuit and escape game between a chasing robot and an escape robot may unfold in diverse map environments, such as street grids, buildings, and underground passages. Related technologies only consider a single map structure to generate pursuit strategies, resulting in a lack of generalization. Therefore, this invention breaks the dependence on specific map structures and types by calculating pursuit and escape strategies separately on multiple training maps. This multi-scenario training mode enables the strategy learning process to capture the patterns of the pursuit and escape game in different maps, laying the foundation for subsequent map generalization and thus meeting the practical needs of dynamic environments in complex public safety scenarios.

[0077] Step S102: Based on the pursuit and escape strategies in all training maps, train the initial strategy model to obtain the target strategy model.

[0078] Specifically, the initial strategy model is iteratively optimized using pursuit and escape strategies generated in different training maps as training data, through reinforcement learning. During the learning process, the model delves into the game logic between the pursuer and the escaper in different maps, while fully considering the diverse actions that the pursuing and escape robots may take in different map environments. This ensures that the final trained target strategy model can accurately generate pursuit strategies in diverse scenarios.

[0079] Step S103: For the target map, use the target strategy model to generate target pursuit strategies for all pursuit robots in the target map.

[0080] Specifically, in practical application scenarios, for any unknown target map, simply input it into the trained target strategy model, and the model can quickly generate a pursuit strategy that is highly adapted to the target map without relying on manual intervention or retraining for new maps. This ensures that the pursuit strategy can be accurately generated in diverse dynamic scenarios, effectively meeting the practical needs of complex environments.

[0081] This invention generates strategies for both the pursuer and the pursuer using adapted algorithms for different types of training maps. By training the model with all training maps and their pursuit strategies, the model learns the strategy features under different map structures, overcoming the limitations of simple map structures and escape strategies. It has zero-shot generalization ability and can directly apply the trained target strategy model to unknown target maps to quickly generate adapted pursuit strategies. These strategies are not easily exploited by adaptive opponents, improving the pursuit success rate in complex game scenarios and meeting the practical needs of complex environments.

[0082] This embodiment provides a method for generating a pursuit strategy in a multi-robot chase game, which specifically includes the following steps:

[0083] Step S201: For each training map, based on the type of the training map, calculate the pursuit strategy of all pursuing robots and the escape strategy of all escaping robots in the training map.

[0084] Specifically, when the training map type is no exit, step S201 above includes:

[0085] Step S2011: Take all possible discrete positions of all pursuing robots in the training map as pursuing nodes and take all possible discrete positions of all escaping robots as escaping nodes. Construct edges based on the positional relationships between all pursuing nodes and escaping nodes to form the graph structure corresponding to the training map.

[0086] Specifically, if the training map has no exit, the game logic of the chase is: the chasing robot chases the escaping robot, and the escaping robot avoids the chasing robot. In this scenario, the training map is discretized into a grid with fixed equal intervals. All grid areas where the chasing and escaping robots may actually be located are the possible discrete positions. These possible discrete positions are abstracted as chasing nodes and escaping nodes, respectively. If any two nodes have directly adjacent reachable paths in the map (such as adjacent cells in the top, bottom, left, or right), then an edge is constructed between these two nodes, thus obtaining the graph structure G corresponding to the training map.<V,E> V represents a node, and E represents an edge.

[0087] Step S2012: Initialize the queue.

[0088] Specifically, an empty queue is initialized to store the game state of the pursuer and the fugitive and the corresponding pursuit steps. The queue follows the first-in, first-out principle.

[0089] Step S2013: For each pursuing node, determine whether the positions of the pursuing node and the escape node satisfy the game function.

[0090] Specifically, the game function f(·) is used to measure whether the pursuing robot can successfully capture the escaped robot, and outputs the corresponding function result based on the positions of the two. For example, when the distance between the pursuing robot and the escaped robot is no more than 1, f(s) = 1, indicating that the game function is satisfied and the pursuing robot can successfully capture the escaped robot; otherwise, f(s) = 0, indicating that the game function is not satisfied. Here, s represents the game state formed by the positions of the pursuing node and the escape node.

[0091] Step S2014: When the positions of the pursuing node and the escape node satisfy the game function, add the positions of the pursuing node and the escape node as a set of game states to the queue, and set the number of pursuit steps for the game state.

[0092] Specifically, when the positions of the pursuing node and the escape node satisfy the game function, that is, when the pursuing robot has the possible positional relationship to capture the escape robot, s = (s p ,s e Store it in the queue and initialize its pursuit steps to 0. Where s p Indicates the location of the pursuit robot, s e This represents the location of the escaped robot. If the two do not satisfy the game function, meaning the current game state cannot directly capture the escapee, then the pursuit steps for this game state are set to ∞.

[0093] Step S2015: When the queue is not empty, retrieve the game state at the head of the queue and determine the game states of the neighboring game states.

[0094] Specifically, when the queue is not empty, the game state of the first robot to enter the queue is retrieved. For the positions of the pursuing robot and the escape robot, their directly adjacent neighbor nodes are determined, and these neighbor nodes are combined to form candidate game states (n). p ,n e ), where n p n represents the neighboring nodes of the pursuing node. e These represent the neighboring nodes of the fleeing node. These candidate game states represent all possible combinations of the next move directions for both the pursuer and the fleeing node from their current positions, covering their potential action options.

[0095] For each n e Enumerate all its neighboring nodes. If the number of pursuit steps in the game state formed by each neighboring node and the currently pursuing node is greater than the number of pursuit steps in the current game state, then it is considered that the escape robot can expand from the current position to n. e The location. For each expandable n e From all the candidate game states it forms, select the candidate game states with a pursuit step count of ∞ (n p ,n e This is used as the neighbor game state. This process ensures that the neighbor game state is obtained by expanding the game state with the largest number of pursuit steps.

[0096] Step S2016: Update the pursuit steps of the neighbor game states based on the game state, add the neighbor game states to the queue, return to the step of taking out the game state at the head of the queue and determining the neighbor game states when the queue is not empty, until the queue is empty, and obtain the pursuit steps of all game states in the graph structure.

[0097] Specifically, for each neighbor game state, its pursuit step count is set to the pursuit step count of the current game state plus 1, and it is added to the queue. Then, returning to step S2015, the pursuit step counts for all game states in the graph structure are finally obtained.

[0098] Step S2017: Calculate the pursuit strategy and escape strategy based on the graph structure.

[0099] In some optional implementations, step S2017 above includes:

[0100] Step a1: Determine multiple first neighbor nodes of each pursuing node and multiple second neighbor nodes of each escape node.

[0101] Specifically, for the current pursuit node, find all its directly adjacent nodes in the graph structure (i.e., adjacent positions that the pursuit robot can move to) as the first neighbor node; at the same time, find all the directly adjacent nodes of the escape node in the graph structure (i.e., adjacent positions that the escape robot can move to) as the second neighbor node.

[0102] Step a2: For each first neighbor node, determine the number of pursuit steps for the game state formed by each second neighbor node and the first neighbor node, and take the maximum value among all pursuit steps as the first target step number corresponding to the first neighbor node.

[0103] Specifically, for each first neighbor node, traverse every second neighbor node of the escaped node to form a game state of (position of the first neighbor node, position of the second neighbor node). Extract the pursuit steps corresponding to the game state from all game states stored in the graph structure. Since there are multiple second neighbor nodes, multiple game states and their corresponding pursuit steps are formed. Select the maximum value among these pursuit steps as the first target step number for the first neighbor node.

[0104] Step a3: Take the minimum value among all the first target steps as the second target step, and take the first neighbor node corresponding to the second target step as the pursuit strategy of the pursuit robot corresponding to the pursuit node.

[0105] Specifically, the minimum number of first target steps among all first neighbor nodes is selected as the second target step number for the current pursuit node. The corresponding first neighbor node is then the optimal movement direction for the pursuit robot, forming the final pursuit strategy. The second target step number represents the minimum number of containment steps (i.e., the fastest containment speed) that the pursuit robot can achieve after considering the optimal response of the escape robot. Choosing the movement direction corresponding to the minimum value reflects the core objective of the pursuit robot to complete the containment at the fastest speed.

[0106] Step a4: For each second neighbor node, determine the number of pursuit steps for the game state formed by each first neighbor node and the second neighbor node, and take the minimum value among all pursuit steps as the third target step number corresponding to the second neighbor node.

[0107] Specifically, for each second neighbor node, all first neighbor nodes of the pursuit robot are traversed to form a game state of (the position of the first neighbor node and the position of the second neighbor node), and the pursuit steps corresponding to the game state are extracted; the minimum value among these pursuit steps is selected as the third target step number of the second neighbor node.

[0108] Step a5: Take the maximum value among all the third target steps as the fourth target step, and take the second neighbor node corresponding to the fourth target step as the escape strategy of the escape robot corresponding to the escape node.

[0109] Specifically, the maximum value among the third objective steps of all second neighbor nodes is selected as the fourth objective step number for the current escape node. The corresponding second neighbor node is then the optimal movement direction for the escaping robot, forming the final escape strategy. The fourth objective step number represents the maximum number of containment steps (i.e., the longest escape time) that the escaping robot can achieve after considering the optimal response of the pursuing robot. Choosing the movement direction corresponding to the maximum value reflects the core objective of the escapee to delay being contained to the greatest extent possible.

[0110] Specifically, when the training map type is one with an exit, step S201 above includes:

[0111] Step S2018: Construct the initial bipartite graph corresponding to the training map, remove nodes without connected edges from the initial bipartite graph, and obtain the target bipartite graph.

[0112] In some optional implementations, step S2018 above, which constructs the initial bipartite graph corresponding to the training map, includes:

[0113] Step b1: Take each exit in the training map as an exit node and each pursuit robot as a pursuit node.

[0114] Specifically, if the training map has exits, the logic of the chase game is as follows: the chasing robot arrives at the exit before the escaping robot, and the escaping robot avoids the chasing robot and reaches an unoccupied exit. In this scenario, each exit is abstracted as an exit node, and the chasing robot is abstracted as a chasing node.

[0115] Step b2: For each pursuit node and each exit node, determine the second shortest distance from the pursuit node to the exit node, and determine the third shortest distance from the escape robot to the exit.

[0116] Specifically, for each pair of pursuit nodes and exit nodes, the shortest path (second shortest distance) for the pursuing robot to move from its current position to the exit is calculated; simultaneously, the shortest path (third shortest distance) for the escaping robot to move from its current position to the exit is calculated. Optionally, the shortest distance can be solved using a shortest path algorithm in a graph structure (such as Dijkstra's algorithm). By quantifying the time difference between the pursuing and escaping robots reaching the exit, an objective basis is provided for determining whether the pursuing robot can effectively intercept the exit.

[0117] Step b3: When the second shortest distance is not greater than the third shortest distance, construct an edge between the pursuit node and the exit node.

[0118] Specifically, if the second shortest distance is less than or equal to the third shortest distance, meaning the pursuing robot can reach the exit faster or at the same time as the escaping robot, then there is a possibility of interception. The pursuing robot can undertake the interception task at the exit, so an edge is constructed between the pursuing node and the exit node.

[0119] Step b4: Construct an initial bipartite graph based on all exit nodes, all pursuit nodes, and all edges.

[0120] Specifically, by constructing a bipartite graph, it visually shows which pursuit robots can intercept which exits.

[0121] In some alternative implementations, if a pursuing node has no edges in the initial bipartite graph (i.e., the distance from the pursuing robot to all exits is greater than that of the escaping robot, making it impossible to effectively intercept any exit), it is removed from the initial bipartite graph. If an exit node has no edges in the initial bipartite graph (i.e., the distance from all pursuing robots to the exit is greater than that of the escaping robot, making the exit a safe exit for the escaping robot), it is removed from the initial bipartite graph, resulting in the target bipartite graph G. b =<(V) exit V pursuer ),E b >. Among them, V exit V represents the exit node. pursuer E represents the target node. b Represents an edge.

[0122] Step S2019: For each exit node in the target bipartite graph, determine the first shortest distance between each exit node and the escape robot, and sort all the first shortest distances in ascending order.

[0123] Specifically, for each exit node in the target bipartite graph, the shortest path between it and the current position of the escape robot is calculated, which is the first shortest distance. Then, the first shortest distances of all exit nodes are arranged in ascending order to quantify the ease with which the escape robot can reach each exit (the shorter the distance, the easier it is to reach it first). This provides a basis for the subsequent selection of exits that need to be defended, ensuring that the resources of the pursuing party are prioritized for the most urgent interception tasks.

[0124] Step S20110: Determine the maximum number of defensible exits. From all sorted first shortest distances, determine the maximum number of defensible first shortest distances and determine the corresponding target exit nodes.

[0125] Specifically, the maximum number of defensible exits represents the number of exits that can be intercepted by the pursuing robots in a timely manner (i.e., arriving earlier than the escaping robots) under the exit priority order. This is achieved by iteratively enumerating l = 1, 2, ... and using a maximum matching algorithm (e.g., the Hungarian algorithm) to calculate the maximum matching m of the first l exit nodes in the target bipartite graph, stopping when m < l. The l at the end of the loop is the maximum number of defensible exits k. From the sorted first shortest distances, the exit nodes corresponding to the first k shortest distances are selected as the target exit nodes that require priority defense.

[0126] Step S20111: For each target exit node, determine the target pursuit node that has a matching relationship with the target exit node from the target bipartite graph.

[0127] Specifically, based on the edges already constructed in the target bipartite graph, a matching algorithm (such as the Hungarian algorithm) is used to assign a corresponding pursuit node to each target exit node, i.e., a target pursuit node, to ensure that each target exit node has a dedicated pursuit robot responsible for defense.

[0128] Step S20112: For each target pursuit node, the pursuit strategy is to go to the exit node that has a matching relationship with the target pursuit node.

[0129] Specifically, for a target pursuit node, it is directed to the exit with a matching relationship to it via the shortest path, ensuring arrival and deployment as quickly as possible. Optionally, if no exit with a matching relationship to the target pursuit node exists, exits with connected edges are identified, and the node is directed to the exit with the smallest first shortest distance among these exits.

[0130] Step S20113: For each other pursuit node besides the target pursuit node, the pursuit strategy is to go to the corresponding exit node based on the first shortest distance after sorting the edges of the target pursuit node.

[0131] Specifically, for other non-target pursuit nodes, priority is given to heading to the exit node that is connected to it and has the smallest third shortest distance, forming a secondary defense line. Through this layered defense strategy, priority interception of key target exit nodes is ensured, while other pursuit robots cover potential risk exits, constructing a comprehensive interception network.

[0132] Step S20114: For the escape robot, it will prioritize going to the removed exit node. If there is no removed exit node, it will go to the corresponding exit node according to the first shortest distance after sorting, as the escape strategy.

[0133] Specifically, the escape robot prioritizes heading to the nearest removed exit node, that is, the exit node that is not listed as a target exit node and is not defended by a dedicated pursuit robot; if all exits are defended, the robot selects the target to move to in ascending order of the third shortest distance to increase the probability of escape.

[0134] In some alternative implementations, Figure 2 This is a flowchart of generating a pursuit strategy in an exit training map according to an embodiment of the present invention, such as... Figure 2 As shown, an initial bipartite graph is first constructed, and nodes without connected edges are removed to obtain the target bipartite graph. The connection relationships between the pursuit nodes and exit nodes are then determined. For the pursuit strategy, if the pursuit node is in the target bipartite graph, it is driven to the exit with a matching relationship along the shortest path; if no matching relationship exists, it is driven to the exit with a connected edge and the smallest first shortest distance. If the pursuit node is not in the target bipartite graph, it is driven to the location of the escape robot along the shortest path. For the escape strategy, if there is a removed exit node, the escape robot is driven to the nearest removed exit along the shortest path; if there is no removed exit node, the escape robot is driven to the nearest unoccupied exit along the shortest path.

[0135] Step S202: Based on the pursuit and escape strategies in all training maps, train the initial strategy model to obtain the target strategy model.

[0136] Specifically, step S202 includes:

[0137] Step S2021: Randomly select any training map from all training maps, input the training map into the initial policy model, and generate a joint policy. The initial policy model includes a feature extractor, encoder, decoder, and pointer network.

[0138] In some optional implementations, step S2021 above inputs the training map into the initial policy model to generate a joint policy, including:

[0139] Step c1: Input the training map into the feature extractor in the initial policy model to generate the state features of each pursuit robot.

[0140] Specifically, the feature extractor first calculates the shortest path between any two nodes in the graph based on the graph structure corresponding to the training map using the Floyd algorithm. If the training map has an exit, it also needs to calculate the shortest path from each node to the exit node. In addition, the number c of each pursuit node is one-hot encoded, so that all the shortest paths and one-hot codes of each node are used together as the state features of that pursuit node.

[0141] Step c2: Input the state features into the encoder to obtain local features.

[0142] Specifically, the state features are embedded into the fully connected layer. (d represents the vector dimension, n represents the number of nodes) After dimensioning, the input is fed into a multi-layer self-attention encoder with an adjacency matrix mask, where each layer of the self-attention network takes the output of the previous layer as the input of the current layer, and finally outputs local features. In each layer, the self-attention mechanism performs weighted aggregation of the current local features, and at the same time limits the attention range through the adjacency matrix mask, only calculating the attention weight for adjacent nodes with edge connections, ignoring irrelevant nodes, and continuously generating the next layer of local features for each pursued node. More specifically, in each layer of the self-attention network, based on the input features of the current layer, the query vector, key vector and value vector of each node are calculated by the following equation (1).

[0143] q i =W Q h i

[0144] k i =W K h i (1)

[0145] v i =W V h i

[0146] Among them, h i q represents node i in the input feature h, where i represents all possible discrete positions of the pursuing robot and all possible discrete positions of the escaping robot in the graph structure corresponding to the training map; i k represents the query vector for node i; i The key vector of node i; v i W represents the value vector of node i; Q W K and W V This represents the learnable weight matrix.

[0147] Then, based on the query vector, key vector and vector dimension, the attention score is calculated by the following equation (2), and the weight is obtained by softmax normalization by the following equation (3).

[0148]

[0149] Among them, u ij q represents the attention score between nodes i and j in the input feature h; i k represents the query vector for node i; j d represents the key vector of node j; d represents the vector dimension.

[0150]

[0151] Among them, w ij u represents the weight between node i and node j in the input feature h; ij represents the attention score between node i and node j in the input feature h; n represents the number of nodes.

[0152] The input features are updated using the adjacency matrix mask and equation (4) to aggregate the effective information of adjacent nodes, thus obtaining the output features of the current layer. After multiple iterations, the output incorporates local features that integrate local topological relationships.

[0153]

[0154] Among them, h' i w represents the output feature corresponding to node i in the input feature h; ij M represents the weights between nodes i and j in the input feature h; ij Represents the element in row i and column j of the adjacency matrix M; v j This represents the value vector of node j.

[0155] Step c3: Input the local features into the decoder to obtain the global features.

[0156] Specifically, the local features output by the encoder are input into a single-layer self-attention decoder without a mask. Referring to the above equations (1)-(3), the normalized weights and value vectors corresponding to each node in the local features are calculated. Then, all nodes in the local features are aggregated through the following equation (5) to obtain global features, so that each pursuit robot can perceive the global situation.

[0157]

[0158] in, Represents global features; n represents the number of nodes; w j Represents the normalized weight of node j; v j This represents the value vector of node j.

[0159] Step c4 involves concatenating the local and global features and inputting them into the pointer network to obtain the training and pursuit strategy for each pursuit robot, thus obtaining the joint strategy.

[0160] Specifically, local and global features are concatenated and projected onto a fully connected layer. (d represents the vector dimension) After dimensioning, a comprehensive feature integrating local details and global situational awareness is formed and input into the pointer network. This network uses the comprehensive feature as a query to calculate the normalized weights of all neighboring nodes of the pursuing robot, representing the probability that the pursuing robot will move to each neighboring node, which serves as the training pursuit strategy for the pursuing robot. The training pursuit strategies of all pursuing robots are then combined to form a joint strategy.

[0161] In some alternative implementations, Figure 3 This is a flowchart of a strategy model according to an embodiment of the present invention, such as... Figure 3 As shown, the training map is input into the feature extractor of the policy model to obtain the state features of the node corresponding to each pursuit robot. Then, the state features are input into the encoder to obtain local features. The local features are input into the decoder to obtain global features. The local and global features are concatenated and input into the pointer network to obtain the training pursuit strategy for each pursuit robot.

[0162] Step S2022: Randomly select the current position of the escape robot and the current position of each pursuing robot, and select the current action of each pursuing robot based on the joint strategy.

[0163] Specifically, the current positions of the escape robot and each pursuing robot are randomly selected to form the current game state s, and this state is input into the joint policy π generated by the initial policy model. θ (·), sample the current action of each hunting robot according to the strategy probability distribution, such as moving to a neighboring node, to obtain the joint action a of all hunting robots.

[0164] Step S2023: For the current position of each pursuing robot, generate the sampling action of the pursuing robot based on the pursuing strategy of the pursuing robot in the training map, and generate the game action of the escaping robot based on the escaping strategy of the escaping robot in the training map.

[0165] Specifically, for each pursuit robot's current location, based on the pursuit strategy μ of that pursuit robot... * Generate the optimal sampling action a * Meanwhile, based on the current position ν of the escape robot * The escape robot's game action b is generated based on the escape strategy. The sampled action serves as the baseline optimal solution for the pursuit strategy, while the game action simulates the escaper's adaptive adversarial behavior. Both provide guidance and adversarial samples for reinforcement learning, ensuring the policy's robustness.

[0166] Step S2024: Based on the current position and sampling action of the pursuing robot and the current position and game action of the escaping robot, generate a reward signal, the next position of the pursuing robot and the next position of the escaping robot.

[0167] Specifically, based on the current location of the pursuit robot s p and sampling action a * The current position of the escape robot. e Given game action b, calculate the next position reached by both players in the graph structure after sampling and game actions, and generate a reward signal r. The reward signal r is 1 only if the next position satisfies the game function; otherwise, r is 0.

[0168] Step S2025: The current position, next position and current action of the pursuing robot, the current position, next position and game action of the escaping robot, and the reward signal are used as a set of training samples.

[0169] Specifically, the training samples are (s,a,b,r,s'), where s represents the current game state, i.e. the current position of the pursuing robot and the current position of the escaping robot; a represents the sampling action of the pursuing robot; b represents the game action of the escaping robot; r represents the reward signal; and s' represents the next game state, i.e. the next position of the pursuing robot and the next position of the escaping robot.

[0170] Step S2026: Return to the step of randomly selecting any training map from all training maps, inputting the training map into the initial policy model, and generating a joint policy until the number of training samples reaches a preset threshold, and calculate the training loss based on all training samples and the joint policy.

[0171] Specifically, repeat steps S2022 to S2025 until the number of training samples reaches a preset threshold. At this point, the training loss is calculated based on all training samples and the joint policy. More specifically, the value loss is calculated using equation (6), the policy loss is calculated using equation (7), and the temperature coefficient loss is calculated using equation (8). All three losses are used as training losses for gradient updates of the corresponding parameters.

[0172]

[0173] Among them, J Q (φ) represents the value loss; Q φ (s,a) represents a state-action value network with learnable network parameters φ; s represents the current game state in the training sample; a represents the current action of the chasing robot in the training sample; s' represents the next game state in the training sample; r represents the reward signal in the training sample. Let α represent the state value under policy entropy regularization, where α represents the learnable entropy regularization coefficient, and γ represents the game's discount factor, reflecting the decay mechanism of the reward signal over time.

[0174]

[0175] Among them, J π (φ) represents the strategy loss; a * This represents the sampling actions of the pursuit robot in the training samples; π θ (s,a) represents a policy network with learnable network parameters θ; β represents the reference policy guidance coefficient, used to balance reference policy regularization and value loss.

[0176]

[0177] Where J(α) represents the temperature coefficient loss; α represents the learnable entropy regularization coefficient; Represents the target entropy, used as a benchmark for pre-defined entropy regularization.

[0178] Step S2027: Based on the training loss, update the model parameters of the initial policy model, return to the step of randomly selecting any training map from all training maps, inputting the training map into the initial policy model, and generating a joint policy, until the model parameters converge to obtain the target policy model.

[0179] Specifically, the parameters of the initial policy model are updated using gradient descent based on the training loss. Steps S2022 to S2026 are repeated until the model parameters converge, resulting in a target policy model that can generalize to different maps.

[0180] Step S203: For the target map, generate target pursuit strategies for all pursuing robots in the target map using the target strategy model. See details below. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0181] This invention generates strategies for both the pursuer and the pursuer using adapted algorithms for different types of training maps. By training the model with all training maps and their pursuit strategies, the model learns the strategy features under different map structures, overcoming the limitations of simple map structures and escape strategies. It has zero-shot generalization ability and can directly apply the trained target strategy model to unknown target maps to quickly generate adapted pursuit strategies. These strategies are not easily exploited by adaptive opponents, improving the pursuit success rate in complex game scenarios and meeting the practical needs of complex environments.

[0182] This embodiment also provides a pursuit strategy generation device for a multi-robot pursuit game. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0183] This embodiment provides a pursuit strategy generation device for multi-robot pursuit and escape game, such as... Figure 4 As shown, it includes:

[0184] The calculation module 401 is used to calculate the pursuit strategy of all pursuing robots and the escape strategy of escaping robots in each training map, based on the type of the training map.

[0185] Training module 402 is used to train an initial policy model based on the pursuit and escape policies in all training maps, and obtain the target policy model.

[0186] The generation module 403 is used to generate target pursuit strategies for all pursuit robots in the target map using a target strategy model.

[0187] In some alternative implementations, the training map is of type no exit;

[0188] The calculation module 401 includes:

[0189] The first construction unit is used to take all possible discrete positions of all pursuing robots in the training map as pursuing nodes and all possible discrete positions of escaping robots as escaping nodes, and construct edges based on the positional relationships between all pursuing nodes and escaping nodes to form a graph structure corresponding to the training map.

[0190] Initialization unit, used to initialize the queue.

[0191] The judgment unit is used to determine whether the positions of the pursuing node and the escape node satisfy the game function for each pursuing node.

[0192] The setting unit is used to add the positions of the pursuing node and the escape node as a set of game states to the queue when the positions of the pursuing node and the escape node satisfy the game function, and to set the number of pursuit steps for the game state.

[0193] The first determining unit is used to extract the game state of the head of the queue when the queue is not empty, and to determine the game states of the neighboring game states of the game state.

[0194] The second determining unit is used to update the number of pursuit steps for the neighbor game states based on the game state, add the neighbor game states to the queue, return to the step of taking out the game state at the head of the queue when the queue is not empty, and determining the neighbor game states of the game state, until the queue is empty, and obtain the number of pursuit steps for all game states in the graph structure.

[0195] The first computing unit is used to calculate the pursuit strategy and escape strategy based on the graph structure.

[0196] In some alternative implementations, the first computing unit includes:

[0197] The first determining sub-unit is used to determine multiple first neighbor nodes of each pursuing node and multiple second neighbor nodes of each escape node.

[0198] The second determining subunit is used to determine the number of pursuit steps for each first neighbor node in the game state formed by each second neighbor node and the first neighbor node, and to take the maximum value among all pursuit steps as the first target step number corresponding to the first neighbor node.

[0199] The third determining subunit is used to take the minimum value among all the first target steps as the second target step, and to take the first neighbor node corresponding to the second target step as the pursuit strategy of the pursuit robot corresponding to the pursuit node.

[0200] The fourth determining subunit is used to determine the number of pursuit steps for each second neighbor node in the game state formed by each first neighbor node and the second neighbor node, and to take the minimum value among all pursuit steps as the third target step number corresponding to the second neighbor node.

[0201] The fifth determining sub-unit is used to take the maximum value among all the third target steps as the fourth target step, and to take the second neighbor node corresponding to the fourth target step as the escape strategy of the escape robot corresponding to the escape node.

[0202] In some alternative implementations, the training map is of the type with an exit;

[0203] The calculation module 401 also includes:

[0204] The second construction unit is used to construct the initial bipartite graph corresponding to the training map, remove nodes without connected edges from the initial bipartite graph, and obtain the target bipartite graph.

[0205] The sorting unit is used to determine the first shortest distance between each exit node and the escape robot for each exit node in the target bipartite graph, and sort all the first shortest distances in ascending order.

[0206] The third determining unit is used to determine the maximum number of defensible exits. It determines the maximum number of defensible first shortest distances from all sorted first shortest distances and determines the corresponding target exit nodes.

[0207] The matching unit is used to determine, for each target exit node, the target pursuit node that has a matching relationship with the target exit node from the target bipartite graph.

[0208] The fourth determining unit is used to determine, for each target pursuit node, the pursuit strategy is to go to the target exit node that is closest to the target pursuit node and has a matching relationship.

[0209] The fifth determining unit is used to determine, for each other pursuing node besides the target pursuing node, the pursuing strategy is to go to the exit node that is connected to the pursuing node and is closest to the pursuing node.

[0210] The sixth determining unit is used to determine, for the escape robot, the priority to go to the removed exit node, and if there is no removed exit node, to go to the exit node closest to the escape robot, as an escape strategy.

[0211] In some alternative implementations, the second building unit includes:

[0212] The sixth determining sub-unit is used to take each exit in the training map as an exit node and each pursuit robot as a pursuit node.

[0213] The seventh determination subunit is used to determine the second shortest distance from the pursuit node to the exit node and the third shortest distance from the escape robot to the exit node for each pursuit node and each exit node.

[0214] The first construction subunit is used to construct an edge between the pursuit node and the exit node when the second shortest distance is not greater than the third shortest distance.

[0215] The second construction subunit is used to construct the initial bipartite graph based on all exit nodes, all pursuit nodes, and all edges.

[0216] In some alternative implementations, training module 402 includes:

[0217] The first generation unit is used to randomly select any training map from all training maps, input the training map into the initial policy model, and generate a joint policy.

[0218] The selection unit is used to randomly select the current position of the escape robot and the current position of each pursuing robot, and select the current action of each pursuing robot based on a joint strategy.

[0219] The second generation unit is used to generate sampled actions of each pursuing robot based on its pursuit strategy in the training map, and to generate game actions of the escaping robot based on its escape strategy in the training map, for each pursuing robot's current position.

[0220] The third generation unit is used to generate a reward signal, the next position of the pursuing robot, and the next position of the escaping robot based on the current position and sampling action of the pursuing robot and the current position and game action of the escaping robot.

[0221] The seventh determining unit is used to take the current position, next position and current action of the pursuing robot, the current position, next position and game action of the escaping robot, and the reward signal as a set of training samples.

[0222] The second computational unit is used to return to the step of randomly selecting any training map from all training maps, inputting the training map into the initial policy model, and generating a joint policy, until the number of training samples reaches a preset threshold, and then calculating the training loss based on all training samples and the joint policy.

[0223] The eighth determination unit is used to update the model parameters of the initial policy model based on the training loss, return to the step of randomly selecting any training map from all training maps, inputting the training map into the initial policy model, and generating a joint policy, until the model parameters converge to obtain the target policy model.

[0224] In some alternative implementations, the initial policy model includes a feature extractor, an encoder, a decoder, and a pointer network;

[0225] The first generation unit includes:

[0226] The first generation subunit is used to input the training map into the feature extractor in the initial policy model to generate the state features of each pursuit robot.

[0227] The second generation subunit is used to input the state features into the encoder to obtain local features.

[0228] The third generation subunit is used to input local features into the decoder to obtain global features.

[0229] The fourth generation sub-unit is used to concatenate local and global features and input them into the pointer network to obtain the training and pursuit strategy for each pursuit robot, thus obtaining the joint strategy.

[0230] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0231] In this embodiment, the pursuit strategy generation device for the multi-robot pursuit game is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0232] This invention also provides a computer device having the above-described features. Figure 4 The device shown is a pursuit strategy generation device for a multi-robot pursuit game.

[0233] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 5 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0234] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0235] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0236] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0237] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0238] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0239] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0240] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0241] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for generating a pursuit strategy of a multi-robot pursuit-evasion game, characterized in that, The method comprises: For each training map, based on the type of the training map, calculating the pursuit strategy of all pursuit robots and the escape strategy of all escape robots in the training map; Based on the pursuit strategy and the escape strategy in all training maps, training an initial strategy model to obtain a target strategy model; For a target map, using the target strategy model to generate a target pursuit strategy of all pursuit robots in the target map.

2. The method of claim 1, wherein, The type of the training map is no exit; The for each training map, based on the type of the training map, calculating the pursuit strategy of all pursuit robots and the escape strategy of all escape robots in the training map, comprises: Taking all possible discrete positions of all pursuit robots in the training map as pursuit nodes and all possible discrete positions of the escape robot as escape nodes respectively, constructing edges based on the positional relationship between all pursuit nodes and the escape nodes to form a graph structure corresponding to the training map; Initializing a queue; For each pursuit node, judging whether the position of the pursuit node and the position of the escape node satisfy a game function; When the position of the pursuit node and the position of the escape node satisfy the game function, adding the position of the pursuit node and the position of the escape node as a set of game states to the queue and setting the pursuit step number of the game state; When the queue is not empty, taking out the first game state in the queue and determining the neighbor game state of the game state; Based on the game state, updating the pursuit step number of the neighbor game state and adding the neighbor game state to the queue, returning to the step of taking out the first game state in the queue and determining the neighbor game state of the game state when the queue is not empty, until the queue is empty, obtaining the pursuit step number of all game states in the graph structure; Based on the graph structure, calculating the pursuit strategy and the escape strategy.

3. The method of claim 2, wherein, The based on the graph structure, calculating the pursuit strategy and the escape strategy, comprises: Determining a plurality of first neighbor nodes of each pursuit node and a plurality of second neighbor nodes of the escape node; For each first neighbor node, determining the pursuit step number of the game state formed by each second neighbor node and the first neighbor node, and taking the maximum value in all pursuit step numbers as a first target step number corresponding to the first neighbor node; Taking the minimum value in all first target step numbers as a second target step number, and taking moving to the first neighbor node corresponding to the second target step number as the pursuit strategy of the pursuit robot corresponding to the pursuit node; For each second neighbor node, determining the pursuit step number of the game state formed by each first neighbor node and the second neighbor node, and taking the minimum value in all pursuit step numbers as a third target step number corresponding to the second neighbor node; Taking the maximum value in all third target step numbers as a fourth target step number, and taking moving to the second neighbor node corresponding to the fourth target step number as the escape strategy of the escape robot corresponding to the escape node.

4. The method of claim 1, wherein, The type of the training map is with exit; The calculating, for each training map, the pursuit strategy of all pursuit robots and the escape strategy of all escape robots in the training map based on the type of the training map further comprises: constructing an initial bipartite graph corresponding to the training map, removing nodes without edges in the initial bipartite graph to obtain a target bipartite graph; determining a first shortest distance between each exit node and the escape robot, and sorting all first shortest distances in ascending order; determining a maximum number of defendable exits, determining the maximum number of defendable exits from all first shortest distances sorted in ascending order, and determining the corresponding target exit nodes; for each target exit node, determining a target pursuit node having a matching relationship with the target exit node from the target bipartite graph; for each target pursuit node, taking the target exit node closest to the target pursuit node and having a matching relationship as the pursuit strategy; for each other pursuit node except the target pursuit node, taking the exit node having an edge with the pursuit node and closest to the pursuit node as the pursuit strategy; for the escape robot, taking the removed exit node as the escape strategy in priority, and taking the exit node closest to the escape robot when there is no removed exit node.

5. The method of claim 4, wherein, The constructing an initial bipartite graph corresponding to the training map comprises: taking each exit in the training map as an exit node, and taking each pursuit robot as a pursuit node; for each pursuit node and each exit node, determining a second shortest distance from the pursuit node to the exit node, and determining a third shortest distance from the escape robot to the exit node; when the second shortest distance is not greater than the third shortest distance, constructing an edge between the pursuit node and the exit node; based on all exit nodes, all pursuit nodes, and all edges, constructing the initial bipartite graph.

6. The method of claim 1, wherein, The training, based on the pursuit strategy and the escape strategy in all training maps, of an initial strategy model to obtain a target strategy model comprises: randomly selecting any training map from all training maps, inputting the training map into the initial strategy model to generate a joint strategy; randomly selecting a current position of the escape robot and a current position of each pursuit robot, and selecting a current action of each pursuit robot based on the joint strategy; for the current position of each pursuit robot, generating a sampled action of the pursuit robot based on the pursuit strategy of the pursuit robot in the training map, and generating a game action of the escape robot based on the escape strategy of the escape robot in the training map; based on the current position and the sampled action of the pursuit robot and the current position and the game action of the escape robot, generating a reward signal, a next position of the pursuit robot, and a next position of the escape robot; taking the current position, the next position, and the current action of the pursuit robot, the current position, the next position, and the game action of the escape robot, and the reward signal as a set of training samples; The step of inputting the training map into the initial strategy model to generate a joint strategy is returned to randomly selecting any training map from all training maps until the number of training samples reaches a preset threshold, and a training loss is calculated based on all training samples and the joint strategy; Based on the training loss, the model parameters of the initial strategy model are updated, and the step of inputting the training map into the initial strategy model to generate a joint strategy is returned to randomly selecting any training map from all training maps until the model parameters converge to obtain the target strategy model.

7. The method of claim 6, wherein, The initial strategy model includes a feature extractor, an encoder, a decoder, and a pointer network; The step of inputting the training map into the initial strategy model to generate a joint strategy includes: Inputting the training map into the feature extractor of the initial strategy model to generate state features of each pursuit robot; Inputting the state features into the encoder to obtain local features; Inputting the local features into the decoder to obtain global features; After splicing the local features and the global features, inputting them into the pointer network to obtain the training pursuit strategy of each pursuit robot and the joint strategy.

8. A pursuit strategy generation device for a multi-robot pursuit-evasion game, characterized by, The device includes: A calculation module configured to, for each training map, calculate the pursuit strategy of all pursuit robots and the escape strategy of all escape robots in the training map based on the type of the training map; A training module configured to train an initial strategy model based on the pursuit strategy and the escape strategy in all training maps to obtain a target strategy model; A generation module configured to, for a target map, generate target pursuit strategies of all pursuit robots in the target map using the target strategy model.

9. A computer device, comprising: It includes: A memory and a processor, which are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the pursuit strategy generation method for multi-robot pursuit and escape game according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for making a computer execute the pursuit strategy generation method for multi-robot pursuit and escape game according to any one of claims 1 to 7.

Images