A phased strategy evaluation method and device for a typical confrontation scene, electronic equipment, storage medium and program product

Abstract

This disclosure provides a method, apparatus, electronic device, storage medium, and program product for phased strategy evaluation in typical adversarial scenarios. The method includes: acquiring competition process information of a target adversarial match, wherein the competition process information describes the adversarial process between two intelligent agents in the target adversarial match; performing phase division processing on the competition process information to obtain phase process information for multiple strategy phases; performing state feature extraction processing on the phase process information of the target strategy phase among the multiple strategy phases to obtain a set of state features for the target strategy phase; and using a trained evaluation model to process the set of state features to obtain the target's strategy evaluation value and feature attribution information for the target agent in the target strategy phase, wherein the feature attribution information represents the causal relationship between multiple state features in the set of state features and the strategy evaluation value. This method can evaluate different phases of an adversarial match with finer granularity.

Description

Technical Field

[0001] This disclosure generally relates to the field of artificial intelligence, and more specifically, to a method, apparatus, electronic device, storage medium, and program product for phased strategy evaluation in typical adversarial scenarios. Background Technology

[0002] Swarm Intelligence Antagonism (also known as Collective Antagonistic Intelligence) is an interdisciplinary research field that integrates swarm intelligence, multi-agent systems, and adversarial games theory. It not only focuses on how to leverage the collaboration of a large number of individuals to achieve complex tasks (such as drone swarming and path planning), but also delves into the competition, interference, and defense mechanisms between different groups. Virtual or real swarm intelligence football and basketball matches, such as virtual swarm versus Google Football and real swarm versus scenario-based football matches, are typical examples of swarm intelligence adversarial sports.

[0003] Evaluating the effectiveness of tactical strategies in swarm intelligence combat sports or scenarios is of great reference value for subsequent targeted review and optimization of tactical strategies.

[0004] However, traditional methods for evaluating adversarial strategies mostly focus on the entire adversarial process, relying solely on the final outcome (win, loss, draw) to assess the strategic performance and ability level of both sides. While this method can evaluate the overall effectiveness of strategy implementation at a macro level, typical adversarial scenarios often involve more detailed tactical variations beneath the macro-tactical framework. For different developments during the adversarial process, both sides will implement more targeted and refined strategies, such as the passing, shooting, and running strategies employed by various agents in a football match. Traditional methods cannot provide targeted analysis and evaluation of these, thus limiting further optimization of tactical strategies. Summary of the Invention

[0005] This disclosure provides a phased strategy evaluation method, apparatus, electronic device, storage medium, and program product for typical adversarial scenarios, to solve at least one of the above-mentioned problems.

[0006] According to a first aspect of the present disclosure, a staged strategy evaluation method for a typical adversarial scenario is provided, comprising: acquiring competition process information of a target adversarial competition, wherein the competition process information is used to describe the adversarial process of two agents in the target adversarial competition; performing stage division processing on the competition process information to obtain stage process information of multiple strategy stages; performing state feature extraction processing on the stage process information of the target strategy stage among the multiple strategy stages to obtain a state feature set of the target strategy stage, wherein the state feature set includes state features of multiple agents of the target side in the target adversarial competition; processing the state feature set using a trained evaluation model to obtain the strategy evaluation value and feature attribution information of the target side in the target strategy stage, wherein the feature attribution information represents the causal relationship between multiple state features in the state feature set and the strategy evaluation value.

[0007] Optionally, the competition process information includes environmental state information, agent state information, and agent action information. The step of dividing the competition process information into stages to obtain stage process information for multiple strategy stages includes: fusing the environmental state information, agent state information, and agent action information to obtain fused information; and dividing the fused information into stages to obtain stage process information for the multiple strategy stages.

[0008] Optionally, the state features include the relationship state features between the plurality of agents and the individual state features of each agent among the plurality of agents.

[0009] Optionally, the relational state features are determined through the following steps: constructing a relational graph of the multiple agents with each agent as a node and the connection between every two agents as edges; and processing the relational graph using a temporal graph attention network to obtain the relational state features.

[0010] Optionally, the staged policy evaluation method further includes: extracting key actions from the stage process information of the target policy stage to obtain a set of key actions for the target policy stage, wherein the set of key actions includes the key actions of the multiple agents in the target policy stage, and the key actions are actions related to the policy of the target policy stage; wherein, processing the state feature set using a trained evaluation model to obtain the policy evaluation value and feature attribution information of the target party in the target policy stage includes: processing the state feature set and the set of key actions using the trained evaluation model to obtain the policy evaluation value and the feature attribution information, wherein the policy evaluation value includes the action evaluation value of each key action in the set of key actions and the stage evaluation value of the target policy stage.

[0011] Optionally, the training sample data of the trained evaluation model includes a set of sample state features and a set of sample key actions as input data, as well as sample action evaluation values ​​and sample stage evaluation values ​​as labels. The set of sample state features is obtained by performing the state feature extraction process on the sample stage process information. The set of sample key actions is obtained by performing the key action extraction process on the sample stage process information. The sample stage process information is obtained by performing the stage division process on the competition process information of the sample adversarial competition. The sample action evaluation value includes the reward value of each sample key action in the set of sample key actions, and the sample stage evaluation value includes the sum of the reward values ​​of all sample actions corresponding to the sample stage process information.

[0012] Optionally, the staged strategy evaluation method further includes: extracting candidate key actions based on the stage process information of the target strategy stage to obtain a set of candidate key actions for the target strategy stage, wherein the set of candidate key actions includes multiple candidate key actions that the multiple agents can execute in the target strategy stage, and the key actions are included in the multiple candidate key actions; wherein, processing the state feature set using the trained evaluation model to obtain the strategy evaluation value and feature attribution information of the target party in the target strategy stage further includes: processing the state feature set and the set of candidate key actions using the trained evaluation model to obtain a candidate action evaluation value for each candidate key action in the set of candidate key actions; and determining recommended key actions based on the candidate action evaluation values ​​of each candidate key action.

[0013] According to a second aspect of the present disclosure, a staged strategy evaluation apparatus for a typical adversarial scenario is provided, comprising: an acquisition unit configured to acquire competition process information of a target adversarial competition, wherein the competition process information is used to describe the adversarial process of two agents in the target adversarial competition; a division unit configured to perform stage division processing on the competition process information to obtain stage process information of multiple strategy stages; an extraction unit configured to perform state feature extraction processing on the stage process information of a target strategy stage among the multiple strategy stages to obtain a state feature set of the target strategy stage, wherein the state feature set includes state features of multiple agents of the target side in the target adversarial competition; and an evaluation unit configured to process the state feature set using a trained evaluation model to obtain a strategy evaluation value and feature attribution information of the target side in the target strategy stage, wherein the feature attribution information represents the causal relationship between multiple state features in the state feature set and the strategy evaluation value.

[0014] Optionally, the competition process information includes environmental state information, agent state information, and agent action information. The division unit is further configured to: perform fusion processing on the environmental state information, the agent state information, and the agent action information to obtain fused information; and perform stage division processing on the fused information to obtain stage process information of the multiple policy stages.

[0015] Optionally, the state features include the relationship state features between the plurality of agents and the individual state features of each agent among the plurality of agents.

[0016] Optionally, the relational state features are determined through the following steps: constructing a relational graph of the multiple agents with each agent as a node and the connection between every two agents as edges; and processing the relational graph using a temporal graph attention network to obtain the relational state features.

[0017] Optionally, the extraction unit is further configured to perform key action extraction processing on the stage process information of the target policy stage to obtain a key action set for the target policy stage, wherein the key action set includes key actions of the plurality of agents in the target policy stage, and the key actions are actions related to the policy of the target policy stage; the evaluation unit is further configured to use the trained evaluation model to process the state feature set and the key action set to obtain the policy evaluation value and the feature attribution information, wherein the policy evaluation value includes the action evaluation value of each key action in the key action set and the stage evaluation value of the target policy stage.

[0018] Optionally, the training sample data of the trained evaluation model includes a set of sample state features and a set of sample key actions as input data, as well as sample action evaluation values ​​and sample stage evaluation values ​​as labels. The set of sample state features is obtained by performing the state feature extraction process on the sample stage process information. The set of sample key actions is obtained by performing the key action extraction process on the sample stage process information. The sample stage process information is obtained by performing the stage division process on the competition process information of the sample adversarial competition. The sample action evaluation value includes the reward value of each sample key action in the set of sample key actions, and the sample stage evaluation value includes the sum of the reward values ​​of all sample actions corresponding to the sample stage process information.

[0019] Optionally, the extraction unit is further configured to perform candidate key action extraction processing based on the stage process information of the target policy stage to obtain a candidate key action set for the target policy stage, wherein the candidate key action set includes multiple candidate key actions that the plurality of agents can execute in the target policy stage, and the key actions are included in the plurality of candidate key actions; the evaluation unit is further configured to: use the trained evaluation model to process the state feature set and the candidate key action set to obtain a candidate action evaluation value for each candidate key action in the candidate key action set; and determine recommended key actions based on the candidate action evaluation values ​​of each candidate key action.

[0020] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform a phased strategy evaluation method for a typical adversarial scenario according to exemplary embodiments of the present disclosure.

[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein instructions in the computer-readable storage medium, when executed by at least one processor, cause at least one processor to perform a phased strategy evaluation method for a typical adversarial scenario according to exemplary embodiments of the present disclosure.

[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by at least one processor, cause at least one processor to perform a phased strategy evaluation method for a typical adversarial scenario according to exemplary embodiments of the present disclosure.

[0023] The technical solutions provided by the embodiments of this disclosure offer at least the following beneficial effects: Compared to the coarse and general evaluations of a complete adversarial match in related technologies, the phased strategy evaluation method, apparatus, electronic device, and storage medium of this disclosure for typical adversarial scenarios, by dividing the match into phases based on complete competition process information, can evaluate different phases of an adversarial match with finer granularity. By extracting a set of state features from the phase process information of a single strategy phase, effective model feature inputs can be provided to the trained evaluation model, thereby evaluating the value of the strategy executed by the agent in each strategy phase in a more targeted manner. Furthermore, by utilizing the model to analyze feature attribution information to reflect the causal relationship between each state feature and the evaluation result, attribution analysis can be achieved. For example, it is possible to more specifically analyze which agent's strategy in which phase affected the development of the entire adversarial situation, providing a clearer reference for further optimization of subsequent tactics and strategies.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0026] Figure 1 This is a flowchart of a phased strategy evaluation method for a typical adversarial scenario according to an exemplary embodiment of the present disclosure.

[0027] Figure 2 This is a schematic diagram of the architecture of a phased strategy evaluation method for a typical adversarial scenario according to a specific embodiment of the present disclosure.

[0028] Figure 3 This is a block diagram of a phased strategy evaluation apparatus for a typical adversarial scenario according to an exemplary embodiment of the present disclosure.

[0029] Figure 4 This is a block diagram of an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0030] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0031] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0032] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0033] The following will describe in detail, with reference to the accompanying drawings, a phased strategy evaluation method and apparatus, electronic device, and storage medium for typical adversarial scenarios according to exemplary embodiments of the present disclosure.

[0034] Figure 1 This is a flowchart of a phased strategy evaluation method for a typical adversarial scenario according to exemplary embodiments of the present disclosure. The method can be executed on an electronic device with sufficient computing power.

[0035] Reference Figure 1 In step S101, the competition process information of the target confrontation is obtained.

[0036] Competition process information is used to describe the adversarial process between the two intelligent agents in a target adversarial competition. It belongs to the low-level information of the adversarial competition and can be collected from the adversarial environment.

[0037] As an example, information about the competition process may include, but is not limited to, real-time environmental state information (such as scores, the position of the ball in a football match, etc.), state information of the agents on both sides (such as the number of agents on each side, the number of each agent, the team to which each agent belongs, etc., as well as real-time information such as the position of the agent, whether it is carrying the ball, and its movement speed), action information of the agent at each step or important node in the competition (i.e., the agent's decision, such as shooting, passing, etc.), and competition time (such as absolute time in the real world, relative time from the start of the game, etc. For relative time, for example, timestamps can be used, or discrete time steps can be used, such as frame numbers).

[0038] In step S102, the competition process information is divided into stages to obtain stage process information for multiple strategy stages.

[0039] Strategy evaluation is typically conducted on one side of an adversarial competition. Therefore, this step can specifically be based on the actions of the agent on the evaluated side (hereinafter referred to as the target side) to divide the competition into stages. As an example, different stage division methods can be formulated based on the type of competition in the target adversarial competition (e.g., including but not limited to football and basketball). For a certain competition type, several typical strategies (e.g., high pressing, low defense, counter-attack in football) can be pre-constructed according to expert-approved rules, and corresponding keywords can be constructed for each typical strategy (e.g., for the goalkeeper's attack tactic, the corresponding keywords can include goalkeeper and receiving the ball). When executing step S102, the actions of the agent on the target side at different times are semantically described, and key fields are extracted from the semantic description as the basis for stage division. These fields are then matched with the keywords of the pre-constructed typical strategies to determine the typical strategies adopted by the target side's agent at each time. Multiple times when the same typical strategy is adopted are then classified into a single strategy stage. As an example, in a complete competitive match, the same typical strategy may appear multiple times. However, if they are not distributed continuously in time, they can be classified into different strategy phases so that the resulting multiple strategy phases are arranged in chronological order. Alternatively, they can be classified into the same strategy phase to reduce the overall number of strategy phases. This disclosure does not impose any restrictions on this. Of course, other reasonable phase division methods can also be adopted, as long as they can achieve an objective phase division of the entire competition process based on clear judgment criteria. This disclosure does not impose any restrictions on this.

[0040] Optionally, for embodiments where the competition process information includes environmental state information, agent state information, and agent action information, step S102 includes: fusing the environmental state information, agent state information, and agent action information to obtain fused information; and performing stage-based processing on the fused information to obtain stage process information for multiple policy stages. By fusing different types of competition process information together, different types of information can complement and corroborate each other, which not only helps to achieve a more comprehensive description of the competition process but also helps to extract more effective state features in subsequent steps.

[0041] As an example, the environmental state information and the agent state information can be time-aligned first. The environmental state information obtained from a competition can be fused with the agent state information to obtain the first type of fused data. Then, the agent action information (i.e., the agent's decision at each step) can be fused with the first type of fused data to obtain the second type of fused data, which serves as the fused information.

[0042] In step S103, the stage process information of the target strategy stage in multiple strategy stages is processed by state feature extraction to obtain the state feature set of the target strategy stage.

[0043] In actual evaluation, only one policy phase can be evaluated at a time, with the target policy phase being the policy phase currently being evaluated. This step involves feature engineering the phase process information of the target policy phase (which, in the embodiment undergoing fusion processing, could be fused information) to form a set of state features for that phase, including the state features of multiple agents of the target side in the objective adversarial competition. The set of state features can also be understood as a state space.

[0044] Optionally, the state features include the state features of relationships between multiple agents and the individual state features of each agent within the multiple agents. By extracting individual state features and relationship state features for each agent and the whole formed by the agents, the state of the target agent can be richly represented from different perspectives, providing more detailed evidence for policy evaluation and richer dimensions for attribution analysis.

[0045] As an example, fused information can be used as a foundation to extract key information (including but not limited to key fields). Specifically, this includes: the tactical strategy type field of the target strategy phase (representing the typical strategy), the agent executing the decision in each frame of the target strategy phase (the time steps within this phase can be determined based on the start and end times of the target strategy phase), the specific actions of the decision, the position of the decision agent in the adversarial field, the positions of other agents, and information about the team each agent belongs to. This allows for the identification of agents directly and indirectly related to the decision agent, thus distinguishing the relationships between different agents when executing each action. By performing secondary calculations on the fused information, the shared state of the agents, i.e., the relational state features, can be obtained. Furthermore, expert knowledge can be used to construct the private state of each agent in different scenarios, obtaining individual state features. Both together constitute the state space of the agent. As an example, a separate state space can be constructed for each agent, which may include, for example, the individual state features of the agent, as well as the portion of the relational state features related to the agent.

[0046] Optionally, for the aforementioned secondary calculation, the relational state features are determined through the following steps: A relational graph is constructed using each agent as a node and the connection between any two agents as edges; the relational graph is then processed using a Temporal Graph Attention Network (TGAT) to obtain the relational state features. The relationships between multiple agents change over time. By constructing the relational graph and utilizing a Temporal Graph Attention Network that integrates graph attention mechanisms and temporal modeling capabilities to process the relational state features, the network can adaptively capture the evolving topological associations and feature dependencies between nodes over time, outputting time-aware relational state features, thereby constructing a graph embedding state shared by multiple agents.

[0047] In step S104, the trained evaluation model is used to process the state feature set to obtain the target's strategy evaluation value and feature attribution information in the target strategy phase.

[0048] It should be understood that the strategy evaluation value is relative to the target party, such as evaluating the attacker's strategy in the target strategy phase. Feature attribution information represents the causal relationship between multiple state features in the state feature set and the strategy evaluation value. Feature attribution information can be used to describe the importance distribution of factors that cause the strategy evaluation value, such as including but not limited to the importance ranking of various state features.

[0049] As an example, when training the model, information from multiple complete adversarial matches extracted from virtual environments, such as data from virtual adversarial platforms with intelligent team adversarial scenarios like Google Football, or information from multiple complete adversarial matches collected from real-world adversarial scenarios, can be used as sample adversarial match information. The trained evaluation model can be obtained through data processing and feature engineering, as well as subsequent steps of stage partitioning and policy evaluation calculation, to design and train the network model. Specifically, a policy evaluation network structure capable of attribution analysis can be designed, relevant initialization parameters defined, and training sample data input into the designed network for training. Regarding the training sample data, after completing the stage partitioning, the policy stages of different types of policies can be classified into different datasets. One type of policy dataset can be selected for model design and training, thereby using the trained evaluation model to perform targeted policy evaluation for the policy stages of that type of policy. In other words, after dividing the target adversarial competition into multiple policy phases in step S102, step S104 can select the trained evaluation model of the corresponding policy type for the target policy phase, perform policy evaluation, and obtain the policy evaluation value of the target policy phase and the corresponding feature attribution information.

[0050] Optionally, the staged strategy evaluation method for typical adversarial scenarios according to the exemplary embodiments of this disclosure further includes: extracting key actions from the stage process information of the target strategy stage to obtain a set of key actions for the target strategy stage, wherein the set of key actions includes key actions of multiple agents in the target strategy stage, and key actions are actions related to the strategy of the target strategy stage; step S104 includes: using a trained evaluation model to process the state feature set and the set of key actions to obtain strategy evaluation values ​​and feature attribution information, wherein the strategy evaluation value includes the action evaluation value of each key action in the set of key actions and the stage evaluation value of the target strategy stage. By extracting the actions actually executed by each agent of the target side from the stage process information of the target strategy stage, and further filtering to obtain key actions related to the strategy of the target strategy stage, evaluating these key actions separately, and evaluating the entire target strategy stage, more detailed evaluation results can be provided, providing richer references for further optimization of subsequent tactics and strategies. The set of key actions can also be understood as an action space.

[0051] As an example, regarding the extraction of key action sets, data statistics and relevant expert knowledge can be used to filter the action space of each agent in the target strategy phase. If there are continuous actions, they need to be discretized and uniformly represented by a discrete action space. In other words, abstract rule-based expert knowledge can be used to filter out the discretized actions that will be executed in different scenarios from all possible actions that the agent can perform, thereby effectively reducing the agent's state space. For agents that need to perform continuous actions, their actions are discretized before constructing their action space, thus keeping the action space of all agents discrete. Correspondingly, the attribution analysis at this time can be to assign the results of the strategy evaluation value to each agent of the target side and related influencing factors to obtain some factors that have a greater impact on the strategy advantages and disadvantages in the target strategy phase. This helps to identify key agents or key decisions that have a greater impact on the situation during the confrontation. Effective evaluation can be achieved for all agents participating in the confrontation, thereby enabling more targeted optimization of subsequent tactical strategies.

[0052] Optionally, the training sample data of the trained evaluation model includes a set of sample state features and a set of sample key actions as input data, as well as sample action evaluation values ​​and sample stage evaluation values ​​as labels. The set of sample state features is obtained by extracting state features from the sample stage process information. The set of sample key actions is obtained by extracting key actions from the sample stage process information. The sample stage process information is obtained by dividing the competition process information of the sample adversarial competition into stages. The sample action evaluation value includes the reward value of each sample key action in the sample key action set, and the sample stage evaluation value includes the sum of the reward values ​​of all sample actions corresponding to the sample stage process information. During the model training phase, when acquiring the competition process information of the sample adversarial competition to construct the training sample data, in addition to acquiring information related to the extraction of the state feature set and key action set, the reward value (also known as the environmental feedback value) obtained by the agent after each step of action execution is also acquired, providing a data foundation for constructing the labels in the training sample data. Specifically, for a sample policy stage, the reward value of a key action can be used as the sample action evaluation value of that key action, and the sum of the reward values ​​of all actions in that stage can be used as the sample stage evaluation value. These can be used as the learning targets for the action evaluation value and stage evaluation value output by the model during training.

[0053] As an example, in a virtual adversarial environment, after an agent executes a decision (action) each frame, the environment automatically provides feedback (rewards) for that decision. The reward value obtained by the agent at each step of a sample policy phase can be recorded and summed progressively, starting from the start time of that phase, to obtain the final phase evaluation value. In a real adversarial environment, the results of the agent's decisions need to be used to design corresponding calculation methods to obtain and record the feedback value of each decision. Similarly, starting from the start time of a sample policy phase, these values ​​are summed progressively to obtain the final phase evaluation value.

[0054] Optionally, the staged strategy evaluation method for typical adversarial scenarios according to the exemplary embodiments of this disclosure further includes: extracting candidate key actions based on the stage process information of the target strategy stage to obtain a set of candidate key actions for the target strategy stage, wherein the set of candidate key actions includes multiple candidate key actions that multiple agents can execute in the target strategy stage, and key actions are contained in multiple candidate key actions; wherein, using a trained evaluation model to process the state feature set to obtain the strategy evaluation value and feature attribution information of the target party in the target strategy stage, it further includes: using a trained evaluation model to process the state feature set and the set of candidate key actions to obtain a candidate action evaluation value for each candidate key action in the set of candidate key actions; and determining recommended key actions based on the candidate action evaluation values ​​of each candidate key action. By extracting other candidate key actions that can be executed at each moment when a key action is actually performed based on the stage process information of the target strategy stage, and calculating their evaluation values, it is possible to evaluate all executable actions at the same moment, thereby determining the most valuable recommended key actions, realizing higher-value strategy recommendation suggestions, and providing richer references for further optimization of subsequent tactics and strategies.

[0055] As an example, as mentioned earlier, the same type of typical strategy may appear multiple times in a complete adversarial game. Based on this, candidate key actions can include all key actions actually executed for the same type of typical strategy. Therefore, in an embodiment that classifies the time periods in which the same type of typical strategy appears at different times into the same strategy phase, the actions in the candidate key action set and the key action set are the same. The difference between the two sets is that each key action in the key action set has a corresponding time (i.e., the time when the key action is actually executed), and the evaluation model needs to output the value of a key action executed at its corresponding time, which is the action evaluation value of the key action; the candidate key action set serves as the source of recommended key actions for each time, and the evaluation model needs to output the value of executing other candidate key actions at the time corresponding to the key action for each key action, which is the candidate action evaluation value.

[0056] As an example, when using a trained evaluation model to process the candidate action evaluation values, for a key action that is included in multiple candidate key actions, the previously calculated action evaluation value can be directly used as its candidate action evaluation value without repeating the calculation, or the calculation can be repeated. This disclosure does not restrict this.

[0057] Next, combine Figure 2 This invention introduces a phased strategy evaluation method for typical adversarial scenarios according to a specific embodiment of the present disclosure. This specific embodiment addresses the limitations of related art strategy evaluation methods, such as limited application scope, inability to objectively evaluate tactical strategies in adversarial scenarios, and the shortcomings of single analysis methods and inability to perform attribution analysis of adversarial results. It enables effective evaluation and attribution analysis of strategies at different tactical strategy stages in typical adversarial scenarios.

[0058] In general, the model design and training steps of this specific embodiment are as follows: The processed and divided different policy stages are used as basic data for model training and testing. Different evaluation models can be designed for different tactical scenarios. A type of tactical scenario is selected as the evaluation object. The dataset of the relevant policy stages is filtered, and the reward value of each agent's single decision in this type of policy stage is used as its label. The evaluation of a single policy is defined as a regression task, and the cumulative reward value of the stage is used as the stage label. The evaluation of the overall policy of the stage is also defined as a regression task. The constructed agent state space and action space are used as feature inputs to this dataset, and the reward value is used as the data label. The agent's single decision value evaluation can be designed as a deep neural network, and the policy stage evaluation can be designed as a decision tree network. The dataset is divided into training set, validation set, and test set according to a certain ratio. The training set and validation set are used for model training, and the test set is used to test the model's performance. The trained evaluation model is stored, and a deep neural network is used to score the agent's single decision. A decision tree model is used to perform attribution analysis using the feature importance ranking method, that is, to rank the importance of features to the model's prediction results, thereby performing attribution analysis.

[0059] Specifically, it includes the following 7 steps.

[0060] Step 1: Obtain information on the competition process of multiple complete matches.

[0061] Specifically, the information obtained for each location includes the environmental state for each frame (each time step). State information of each adversarial agent in each frame ,Right now and ,in Represents the intelligent agent of the opposing side. Information, , The number of intelligent agents in this direction. Indicating confrontation with the other side, , The number of the other party's agents; the decision made by each agent per frame. i.e., intelligent agent exist The action taken at any given moment; the reward (feedback value) received by the agent after executing the decision. .

[0062] It should be noted that virtual and real-world competitive environments have different reward value acquisition methods. In multi-agent virtual competitive environments such as Google Soccer, the environment automatically provides corresponding rewards after an agent makes a decision; the reward for each decision can be directly saved. For typical real-world team-based competitive scenarios such as soccer matches, corresponding decision feedback methods need to be designed based on the different strategies to be evaluated. The following section uses the more complex real-world soccer match scenario as an example to detail how this specific implementation calculates the reward value obtained by the agent (player) for each decision in a real-world competitive scenario. Scoring is the most important evaluation criterion for measuring the results of player decisions (actions) in soccer matches, and it is also applicable to other team-based competitive sports scenarios.

[0063] Specifically, the court is divided into There are 3 grids, each representing a possible position of the ball. and Depending on the specific needs, several different probabilities can be calculated for each grid cell: (1) When the player with the ball (the decision-making agent) is in position At that time, the probability of deciding to dribble or pass the ball to other positions. (2) When the player with the ball is in position The probability of a direct shot (3) When the player with the ball is in position Move to another grid probability (4) When the player with the ball is in position The probability of scoring when shooting. As an example, these probabilities can be derived from statistical data, such as statistics on player i's position in this match or multiple historical matches. The probability of choosing to shoot directly is calculated as "Number of shots / (Number of shots + Number of moves)". The calculation of other probabilities is similar; these probabilities can also be obtained directly by inputting feature data into a relevant model, and this disclosure does not impose any restrictions on this. After obtaining these probabilities, a series of cooperative decisions (actions) between players are regarded as a Markov chain that can be iteratively calculated. Taking the final shot and goal as the scoring basis, the value of each grid is calculated, and the value of each decision can be iterated backwards. Specifically, backwards refers to the temporal backwards, with the final shot as the ending point, and the action of the agent is executed backwards according to the temporal order. That is, if the player with the ball moves the ball from... Move to Known grid The value is Then the grid The value is:

[0064] .

[0065] In the above formula, The value is generally the probability of a shot and a goal, which is the initial value for iterative calculation. It can be obtained by statistically analyzing data from multiple matches, specifically from player decision-making data obtained from the competition process. The value obtained from each grid cell is assigned to the player who made the relevant decision as the value of that decision.

[0066] Step 2: After fusing and processing the information of the competition process, the complete adversarial process is divided into stages, the adversarial competitions to be evaluated for strategy are selected, and the required dataset is filtered out.

[0067] The environmental state obtained from a game With agent information The data is then cross-completed through fusion to obtain the first type of fused data. Then, the agent's decision-making information is combined with the first type of fused data. Cross-information completion is performed to obtain the second type of fused data. The key fields in the agent's decision-making information are retrieved as the necessary criteria for segmentation, dividing the entire adversarial process into different strategy phases. ,in This marks the start time of the first phase. The end time of a phase, in order to To obtain the corresponding data stage for the retrieval criteria , A type of adversarial scenario is selected for strategy evaluation. Based on the different scenario categories, this type of adversarial scenario is selected from all collected adversarial data as the base dataset.

[0068] Taking a football match as an example, data recording environmental information (match data) and data recording the position and speed of players at every moment during the match (tracking data) are merged to correct missing or incomplete data fields such as player names and numbers in the tracking data, resulting in merged tracking data. Data recording player decision-making information (event data) is then merged with the tracking data. This involves locking the position of each event data point in the tracking data by frame count, identifying the player executing the decision (the player with the ball), and supplementing the corresponding event data with the position and speed information of players without the ball (all players except the player executing the decision). This expands the event data, including not only the position, decision type, and position and speed of the player with the ball, but also the position and speed of all other players and the ball at that moment, resulting in advanced fusion information. Relevant keywords are extracted from the advanced fusion information for stage division. For example, the "possession" field can divide a complete match into the possession phases of both sides and the start time of each phase, while the "counterpressing" field can be used to divide the "counter-pressing" strategy phase. The corresponding data phase is extracted by capturing the frame count at the beginning and end of each phase. This allows us to select a particular strategy scenario for evaluation.

[0069] Step 3: Extract key fields from the fusion information of each stage to construct the state space of the agent.

[0070] Specifically, the start time of each phase of a certain type of tactic in a confrontation. and end time The agent that executes the action in each step (each frame). The specific actions to be performed A coordinate system is established for the adversarial scenario. A three-dimensional coordinate system can be used, or a polar coordinate system can be used if the adversarial scenario is in a curved space. The coordinates of each agent in each frame are extracted. Using agents as nodes and the relationships between agents as edges, a feature set of nodes and edges is constructed. A temporal graph attention network is used to perform heterogeneous graph modeling for the tactical phase to obtain the graph embedding states that agents can share. Then, the quantized expert knowledge set is used. For each policy phase to be evaluated, a private state is constructed for each agent. The shared state and the private state together constitute the state space of the agent.

[0071] Taking a football match as an example, to evaluate the efficiency of the attacking team's strategy, node features need to be calculated as follows: whether the player has the ball, the distance from the node to the ball, the distance from the node to the center of the goal, the number of defenders within a 5-meter radius, the number of attackers, whether the node is in the defender's penalty area, whether it is a defender, and the node's role, etc.; edge features need to be calculated as follows: whether two nodes are on the same team, the distance between them, the angle, and whether a pass has occurred between the nodes, etc. A temporal graph attention network is used to construct the shared graph embedding state of the agents, represented as... Then, the scenario-dependent expert knowledge is abstracted into several computable rule indicators. Calculate per frame Advanced features, such as the mean y-coordinate of the defending player, the width of the defending team's x-coordinate, the density center of the defending team calculated using K-means, whether the current frame belongs to the first or second half, whether the current frame is under pressure in the event data, and whether the current frame is in the counter-pressing phase, can also include the current agent's position coordinates, distances to other agents, distances to the goal, and distances to the ball, to obtain the agent's private state, which can be represented as... Then, at time t, the state space of node i is represented as follows: .

[0072] Step 4: Calculate the action space of each agent.

[0073] Depending on the execution strategy at different policy stages, the agent's full action space needs to be filtered to include only those actions directly related to the current policy stage. This improves model training efficiency and enhances action evaluation accuracy. Furthermore, since the action space is discrete, continuous actions such as running need to be discretized to maintain action consistency.

[0074] Taking a football match as an example, for players in an offensive scenario, it is necessary to filter out the defensive actions that the player with the ball will inevitably take under certain circumstances, and only select actions related to offense to constitute their action space. For players running without the ball, it is necessary to discretize their continuous running into running strategies in different directions before calculating the reward value: First, the player's running direction is divided into 5 categories (forward, backward, left, right, and stationary). Taking the player's current coordinates as the center, the four standard directions are forward, backward, left, and right, and the angle is calculated in a clockwise direction. The running direction (-45°, 45°] is defined as forward, (45°, 90°] as right, and so on. The running direction vector between the current frame and the next frame is used to determine the running strategy of the current frame. If there is no obvious intention to move, it is determined to be stationary.

[0075] Step 5: Calculate the reward value obtained by the agent after executing the action policy, and the total reward value for each adversarial phase.

[0076] Specifically, in a virtual adversarial environment, the reward obtained by the agent at each step in each phase is recorded as... The start time of this stage is The end time is ,in The final evaluation feedback value obtained at this stage is... For a real-world adversarial environment, the reward value of the key events in each phase must first be calculated. The evaluation feedback values ​​of these events are then summed to obtain the overall reward value for that phase.

[0077] Taking a football match as an example, using the player decision value calculation method described in step 1, all the reward values ​​(values ​​in different positions) obtained by the player with the ball in a phase are accumulated sequentially according to the frame number to obtain the overall reward system for each confrontation phase.

[0078] Step 6: Design the corresponding evaluation model based on the specific competitive match to be evaluated, and train the model.

[0079] Specifically, using the agent's state space, action space, and reward obtained in the preceding steps, a sample set is constructed for model learning. Each frame contains all the agent's states, actions, and rewards as a single data point. Using a reinforcement learning framework, an evaluation of the agent's single-step actions is designed. By inputting all policy stages to be evaluated into the model, an action evaluation value for each action in that stage can be obtained. The model can be designed as a deep neural network model, requiring the following network parameters: loss function, number of hidden layers, number of neurons in each hidden layer, activation function of each neuron, initial input, batch training sample size, and random seed. Each frame contains all the agent's states and actions as a single data point, and the overall reward value for that stage is used as the label. This policy task is defined as a regression task, aiming to obtain a stage evaluation value for each policy stage after inputting it into the model. The model can be designed as a decision tree model or a deep neural network. If the model is designed as a decision tree, the LightGBM (Light Gradient Boosting Machine) algorithm is preferred. The network parameters that need to be set include: loss function, number of decision tree layers, number of leaf nodes, maximum number of iterations, random seed, number of samples per leaf node, and sample weights. Other relevant parameters can be set to default values. If the model is designed as a deep neural network, the network parameters that need to be set include: loss function, number of hidden layers, number of neurons in each hidden layer, activation function of each neuron, initial input, batch training sample size, and random seed.

[0080] Divide the dataset into training, validation, and test sets. The ratio of these sets can be set according to the actual situation, such as 7:2:1 or 8:1:1. Store the converged model.

[0081] Step 7: Input the target strategy phase of the target adversarial game to be evaluated into the trained evaluation model to obtain its strategy evaluation value and perform attribution analysis.

[0082] Specifically, the strategy stage to be evaluated is input into the evaluation model, and the value calculated by the model is the evaluation value of the strategy adopted for this strategy stage. If the model is a decision tree model, the influence order of different features on the final result can be obtained by using the feature importance ranking function of the model, thus obtaining the attribution analysis of the strategy effectiveness. If the model uses a deep neural network, the feature importance can be ranked by methods such as OOB (Out-of-Bag) to perform attribution analysis.

[0083] This specific embodiment effectively divides different strategy stages in a typical adversarial scenario. By performing targeted feature engineering on the data from each strategy stage, it obtains a heterogeneous state space for the agent under different states, an action space more closely resembling the tactical scenario, and corresponding rewards. For the strategy stage to be evaluated, a corresponding evaluation model is designed. By inputting the state space and action space of the strategy stage into the model, an evaluation value and attribution analysis of the strategy's effectiveness can be obtained, providing reliable support for analysis, review, and strategy optimization. Compared to related technologies that only evaluate based on the adversarial results or subjective, experience-based stage analysis methods, the strategy evaluation method provided in this specific embodiment offers a more targeted evaluation of the implementation strategies at different strategy stages in an adversarial match. It effectively utilizes various types of data collected during the adversarial process, realizing strategy evaluation and attribution analysis of strategy effectiveness. It enables multi-dimensional and more granular evaluation for optimizing and improving tactical strategies.

[0084] Figure 3 This is a block diagram of a phased strategy evaluation apparatus for a typical adversarial scenario according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 3 The phased strategy evaluation device 300 for typical adversarial scenarios includes an acquisition unit 301, a division unit 302, an extraction unit 303, and an evaluation unit 304.

[0085] The acquisition unit 301 can acquire the competition process information of the target adversarial competition, wherein the competition process information is used to describe the adversarial process of the two intelligent agents in the target adversarial competition.

[0086] The segmentation unit 302 can perform stage segmentation processing on the competition process information to obtain stage process information of multiple strategy stages.

[0087] The extraction unit 303 can perform state feature extraction processing on the phase process information of the target strategy phase in multiple strategy phases to obtain the state feature set of the target strategy phase. The state feature set includes the state features of multiple agents of the target side in the target adversarial game.

[0088] Evaluation unit 304 can use a trained evaluation model to process the set of state features and obtain the policy evaluation value and feature attribution information of the target party in the target policy stage. The feature attribution information represents the causal relationship between multiple state features in the set of state features and the policy evaluation value.

[0089] Optionally, the competition process information includes environmental state information, agent state information, and agent action information. The partitioning unit 302 may also: perform fusion processing on the environmental state information, agent state information, and agent action information to obtain fused information; and perform stage partitioning processing on the fused information to obtain stage process information for multiple policy stages.

[0090] Optionally, the state features include the state features relating to the multiple agents and the individual state features of each agent among the multiple agents.

[0091] Optionally, the relational state features are determined through the following steps: constructing a relational graph of multiple agents with each agent as a node and the connection between every two agents as edges; and processing the relational graph using a temporal graph attention network to obtain the relational state features.

[0092] Optionally, the extraction unit 303 can also perform key action extraction processing on the stage process information of the target policy stage to obtain a key action set for the target policy stage. The key action set includes key actions of multiple agents in the target policy stage, and the key actions are actions related to the policy of the target policy stage. The evaluation unit 304 can also use a trained evaluation model to process the state feature set and the key action set to obtain policy evaluation value and feature attribution information. The policy evaluation value includes the action evaluation value of each key action in the key action set and the stage evaluation value of the target policy stage.

[0093] Optionally, the training sample data of the trained evaluation model includes a set of sample state features and a set of sample key actions as input data, as well as sample action evaluation values ​​and sample stage evaluation values ​​as labels. The set of sample state features is obtained by extracting state features from the sample stage process information. The set of sample key actions is obtained by extracting key actions from the sample stage process information. The sample stage process information is obtained by dividing the competition process information of the sample adversarial competition into stages. The sample action evaluation value includes the reward value of each sample key action in the sample key action set, and the sample stage evaluation value includes the sum of the reward values ​​of all sample actions corresponding to the sample stage process information.

[0094] Optionally, the extraction unit 303 may also perform candidate key action extraction processing based on the stage process information of the target policy stage to obtain a candidate key action set for the target policy stage. The candidate key action set includes multiple candidate key actions that multiple agents can execute in the target policy stage, and key actions are contained in multiple candidate key actions. The evaluation unit 304 may also: use a trained evaluation model to process the state feature set and the candidate key action set to obtain a candidate action evaluation value for each candidate key action in the candidate key action set; and determine recommended key actions based on the candidate action evaluation values ​​of each candidate key action.

[0095] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0096] Figure 4 A structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0097] Reference Figure 4 The electronic device 400 includes at least one memory 401 and at least one processor 402. The at least one memory 401 stores computer-executable instructions that, when executed by the at least one processor 402, cause the at least one processor to perform the target correspondence method as described in the exemplary embodiments above.

[0098] As an example, electronic device 400 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 400 is not necessarily a single electronic device 400, but may be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 400 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device 400 locally or remotely (e.g., via wireless transmission) through an interface.

[0099] In electronic device 400, processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 402 may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0100] The processor 402 can execute instructions or code stored in the memory 401, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transmission protocol.

[0101] The memory 401 may be integrated with the processor 402, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 401 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 401 and the processor 402 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 402 to read files stored in the memory.

[0102] In addition, the electronic device 400 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 400 can be interconnected via a bus and / or network.

[0103] According to exemplary embodiments of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein the instructions, when executed by at least one processor, cause at least one processor to perform the target-correspondence method as described in the exemplary embodiments above. Examples of computer-readable storage media herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0104] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, including computer instructions that, when executed by at least one processor, perform the target correspondence method as described in the exemplary embodiments above.

[0105] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0106] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A phased strategy evaluation method for typical adversarial scenarios, characterized in that, include: Acquire competition process information of a target adversarial game, wherein the competition process information is used to describe the adversarial process of the two intelligent agents in the target adversarial game, the competition process information includes environmental state information, agent state information, agent action information and adversarial time, the environmental state information includes at least one of the following: score, ball position in a ball game, the agent state information includes at least one of the following: basic information, real-time information, the basic information includes at least one of the following: number of agents on each side of the adversarial game, agent number, agent to which adversary side, the real-time information includes at least one of the following: agent position, whether the agent is carrying the ball in a ball game, agent movement speed, the agent action information includes at least one of the following: scoring action in a ball game, passing action in a ball game, the adversarial time includes at least one of the following: absolute time in the real world, relative time from the start of the target adversarial game, the relative time includes at least one of the following: timestamp, discrete time step; The competition process information is divided into stages to obtain stage process information for multiple strategy stages; The phase process information of the target strategy phase in the multiple strategy phases is processed by state feature extraction to obtain the state feature set of the target strategy phase, wherein the state feature set includes the state features of multiple agents of the target side in the target adversarial competition. The key action extraction process is performed on the phase process information of the target policy phase to obtain the key action set of the target policy phase. The key action set includes the key actions of the multiple agents in the target policy phase. The key actions are actions related to the policy of the target policy phase. Using a trained evaluation model, the state feature set and the key action set are processed to obtain the strategy evaluation value and feature attribution information of the target party in the target strategy stage. The strategy evaluation value includes the action evaluation value of each key action in the key action set and the stage evaluation value of the target strategy stage. The feature attribution information represents the causal relationship between multiple state features in the state feature set and the strategy evaluation value.

2. The phased strategy evaluation method as described in claim 1, characterized in that, The process of dividing the competition process information into stages yields stage process information for multiple strategy stages, including: The environmental state information, the agent state information, and the agent action information are fused to obtain fused information; The fused information is divided into stages to obtain the stage process information of the multiple strategy stages.

3. The phased strategy evaluation method as described in claim 1, characterized in that, The state features include the relationship state features between the multiple agents and the individual state features of each agent. The relationship state features are determined through the following steps: Using each of the multiple agents as a node and the connection between every two agents as an edge, construct a relationship graph of the multiple agents; The relationship graph is processed using a temporal graph attention network to obtain the relationship state features.

4. The staged strategy evaluation method according to any one of claims 1 to 3, characterized in that, The training sample data of the trained evaluation model includes a set of sample state features and a set of sample key actions as input data, as well as sample action evaluation values ​​and sample stage evaluation values ​​as labels. Specifically, the sample state feature set is obtained by extracting state features from the sample stage process information; the sample key action set is obtained by extracting key actions from the sample stage process information; and the sample stage process information is obtained by dividing the competition process information of the sample adversarial competition into stages. The sample action evaluation value includes the reward value of each sample key action in the sample key action set, and the sample stage evaluation value includes the sum of the reward values ​​of all sample actions corresponding to the sample stage process information.

5. The staged strategy evaluation method according to any one of claims 1 to 3, characterized in that, Also includes: Based on the phase process information of the target policy phase, candidate key actions are extracted to obtain a set of candidate key actions for the target policy phase. The set of candidate key actions includes multiple candidate key actions that the multiple agents can execute in the target policy phase, and the key actions are included in the multiple candidate key actions. Using the trained evaluation model, the state feature set and the candidate key action set are processed to obtain the candidate action evaluation value for each candidate key action in the candidate key action set. Based on the candidate action evaluation values ​​of each candidate key action, the recommended key actions are determined.

6. A phased strategy evaluation device for typical adversarial scenarios, characterized in that, include: The acquisition unit is configured to acquire competition process information of a target competitive match. The competition process information describes the competitive process between the two intelligent agents in the target competitive match. The competition process information includes environmental state information, agent state information, agent action information, and competition time. The environmental state information includes at least one of the following: score, ball position in a ball game. The agent state information includes at least one of the following: basic information, real-time information. The basic information includes at least one of the following: the number of agents on each side, agent ID, and the opposing side to which the agent belongs. The real-time information includes at least one of the following: agent position, whether the agent is carrying the ball in a ball game, and agent movement speed. The agent action information includes at least one of the following: scoring action in a ball game, passing action in a ball game. The competition time includes at least one of the following: absolute time in the real world, relative time from the start of the target competitive match, and relative time including at least one of the following: timestamp, discrete time step. The segmentation unit is configured to perform stage segmentation processing on the competition process information to obtain stage process information of multiple strategy stages; The extraction unit is configured to perform state feature extraction processing on the phase process information of the target strategy phase among the multiple strategy phases to obtain a state feature set of the target strategy phase, wherein the state feature set includes the state features of multiple agents of the target side in the target adversarial competition. The extraction unit is further configured to perform key action extraction processing on the stage process information of the target policy stage to obtain a set of key actions for the target policy stage, wherein the set of key actions includes the key actions of the multiple agents in the target policy stage, and the key actions are actions related to the policy of the target policy stage. An evaluation unit is configured to use a trained evaluation model to process the set of state features and the set of key actions to obtain the strategy evaluation value and feature attribution information of the target party in the target strategy stage. The strategy evaluation value includes the action evaluation value of each key action in the set of key actions and the stage evaluation value of the target strategy stage. The feature attribution information represents the causal relationship between multiple state features in the set of state features and the strategy evaluation value.

7. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. When the computer-executable instructions are executed by the at least one processor, they cause the at least one processor to execute the phased strategy evaluation method for typical adversarial scenarios as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the processor to perform the phased strategy evaluation method for a typical adversarial scenario as described in any one of claims 1 to 5.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by at least one processor, they cause the at least one processor to perform the phased strategy evaluation method for a typical adversarial scenario as described in any one of claims 1 to 5.

Images