An intelligent strategy evaluation method for a confrontation game

Abstract

The application belongs to the technical field of aviation information processing, and particularly relates to an intelligent strategy evaluation method for confrontation game. The method comprises the following steps: S1, calculating the mutual winning rates between each intelligent strategy based on the confrontation results between the collected intelligent strategies; S2, calculating the game scores of each intelligent strategy based on the Elo algorithm after each confrontation between the intelligent strategies ends; S3, calculating the multiple average game scores of each intelligent strategy changing over time based on the moving average method; S4, performing data statistics by using multiple comprehensive evaluation indexes based on the situation data of the confrontation process between the intelligent strategies; and S5, displaying the winning rates, game scores, average game scores and data statistics results in a visual manner. The application can scientifically and quantitatively analyze the advantages and disadvantages of the confrontation intelligent strategies, and provide data support for the improvement of the intelligent strategies.

Description

Technical Field

[0001] This application belongs to the field of aviation information processing technology, and specifically relates to an intelligent strategy evaluation method for adversarial game. Background Technology

[0002] Artificial intelligence (AI) technology is developing rapidly, significantly impacting human society and the world. The intelligent development of future adversarial strategies is particularly prominent in the adversarial field. The Alpha AI strategy developed by the Department of Aeronautical Engineering at the University of Cincinnati, using the Genetic Fuzzy Tree Algorithm (GFT), defeated experienced retired U.S. Air Force Colonel Keane Lee in a simulation environment. The DARPA's Alpha Dogfight project, employing reinforcement learning, defeated human pilots with a resounding 5-0 victory in a simulation environment. Currently, various intelligent technologies are applied to adversarial game problems, optimizing and training diverse intelligent strategies in different adversarial simulation environments. However, research on evaluation methods for intelligent strategies in adversarial game problems is still in its early stages, mainly facing the following problems: a) The quality of intelligent adversarial strategies is judged solely based on win rate, resulting in a relatively singular and incomplete evaluation metric; b) Historical data from adversarial games is lacking in utilization and analysis, failing to comprehensively evaluate the performance of intelligent strategies throughout the adversarial process; c) A suitable method is lacking for tracking and visualizing the capability changes of various intelligent strategies during training and optimization. Summary of the Invention

[0003] To address the aforementioned issues, this application proposes an intelligent strategy evaluation method for adversarial games, which comprehensively evaluates intelligent strategies in adversarial games and assists in the testing, comparison, and optimization of intelligent strategies.

[0004] The intelligent strategy evaluation method for adversarial games provided in this application mainly includes:

[0005] Step S1: Based on the collected adversarial results between various intelligent strategies, calculate the mutual win rate between various intelligent strategies;

[0006] Step S2: After each confrontation between the various intelligent strategies, calculate the game score of each intelligent strategy based on the Elo algorithm.

[0007] Step S3: Calculate the time-varying average game integrals of each intelligent strategy based on the moving average method.

[0008] Step S4: Based on the situational data of the adversarial process between various intelligent strategies, multiple comprehensive evaluation indicators are used for data statistics;

[0009] Step S5: Use a matrix to represent the win rates between each intelligent strategy, use a curve graph to represent the trend of the average game score of each intelligent strategy over time, use a bar chart to represent the final game score ranking of each intelligent strategy, and use a radar chart to represent the statistical results of each comprehensive evaluation index.

[0010] Preferably, step S1 further includes:

[0011] Step S11: Use a computing cluster to conduct large-scale adversarial tests in parallel. In the simulation environment, conduct one-on-one adversarial games between various intelligent strategies. According to the round-robin tournament system, under random initial conditions, any two intelligent strategies will play adversarial games for no less than a set number of rounds. The result of the win or loss between intelligent strategies is set to three situations: victory, defeat, and draw.

[0012] Step S12: Use a structured database to store the game process data and game results;

[0013] Step S13: Calculate the win rates between each intelligent strategy.

[0014] Preferably, in step S11, the number of sessions is set to 1000.

[0015] Preferably, in step S13, when calculating the win rates between various intelligent strategies, the win rate of intelligent strategy A against intelligent strategy B is calculated using the following formula:

[0016]

[0017] Preferably, in step S2, calculating the game integrals of each intelligent strategy based on the Elo algorithm includes:

[0018]

[0019]

[0020] R' A =R A +K(S A -E A );

[0021] R' B =R B +K(S B -E B );

[0022] Among them: E A Let E be the expected win rate of intelligent strategy A. B Let R be the expected win rate of intelligent strategy B. A For the current integral of intelligent policy A, RB R' is the current integral of intelligent policy B. A R' is the integral after adversarial action against intelligent policy A. B Let S be the integral after adversarial action against intelligent policy B, where K is a constant coefficient and S is the integral. A With S B The value is the win / loss score of the intelligent strategy, with 1 for a win, 0.5 for a draw, and 0 for a loss.

[0023] Preferably, step S3 further includes:

[0024] Step S31: Record the game score of each intelligent strategy after each set number of confrontations.

[0025] Step S32: Take a window length of T = 10, and denote the game integral at time t as A. t Then, the average game integral F of intelligent strategy A is calculated using the moving average method. t When using this formula:

[0026]

[0027] Preferably, in step S4, based on the situational data of the adversarial process between various intelligent strategies, data statistics are performed using multiple comprehensive evaluation indicators, including:

[0028] Informational indicators are determined based on radar scan tracking time data between various intelligent strategies.

[0029] Energy indicators are determined based on the integral data of kinetic and potential energy over time for each intelligent strategy.

[0030] The offensive metrics are determined based on the number of times each intelligent strategy knocks down opponents in a winning match.

[0031] Security indicators are determined based on the number of times each intelligent strategy is defeated by the opponent in a losing match.

[0032] Robustness metrics are determined based on the ratio of the lowest to the highest win rates of each intelligent strategy against other intelligent strategies.

[0033] Preferably, in step S5, the informational index, energy index, strikeability index, security index, and robustness index are normalized to between 0 and 1, and radar charts are drawn using the five axes corresponding to the above indexes (informational, energy, strikeability, security, and robustness), and then superimposed together for comparison and display.

[0034] Preferably, step S5 further includes a visual analysis of the reasons for failure, which includes using a dynamically scrolling bar chart to display statistics on the reasons for failure of each strategy during the round-robin tournament.

[0035] Preferably, after step S5, the method further includes:

[0036] Based on the collected adversarial results between various intelligent strategies, the total win rate of each intelligent strategy relative to other intelligent strategies is calculated and normalized to obtain the first evaluation parameter.

[0037] The game integrals of each intelligent strategy are normalized to obtain the second evaluation parameter;

[0038] The first evaluation parameter and the second evaluation parameter are weighted using a weighted average method to obtain the total score of each intelligent strategy.

[0039] This application has the following advantages:

[0040] (1) A set of intelligent strategy evaluation methods applicable to adversarial games has been established, which can scientifically and quantitatively analyze the advantages and disadvantages of adversarial intelligent strategies, filling the gap in this field.

[0041] (2) Real-time monitoring of the capabilities of intelligent strategies in adversarial games has been achieved, focusing on tracking changes in the capabilities of current strategy algorithms and multi-dimensional evaluation.

[0042] (3) The round-robin tournament used for data collection can be conducted in multiple rounds. After a round-robin tournament is completed, a points ranking, multi-dimensional evaluation indicators and situation data records during the confrontation process will be generated. By analyzing and learning these data, strategies can be improved. After the strategy is improved, the next round-robin tournament can be participated in. In this way, all participating strategies can be continuously improved and improved together.

[0043] (4) The comprehensive evaluation index of intelligent strategies in adversarial games aims to analyze the results of different strategies when they are in mutual confrontation and the process of strategies in the whole confrontation, so as to assist researchers in analyzing the advantages and disadvantages of strategies. Attached Figure Description

[0044] Figure 1 This is a flowchart of a preferred embodiment of the intelligent strategy evaluation method for adversarial games according to this application.

[0045] Figure 2 This is a visualization of a preferred embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are only some, not all, of the embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0047] The first aspect of this application provides an intelligent strategy evaluation method for adversarial games, such as... Figure 1 As shown, it mainly includes:

[0048] Step S1: Based on the collected adversarial results between various intelligent strategies, calculate the mutual win rate between various intelligent strategies;

[0049] Step S2: After each confrontation between the various intelligent strategies, calculate the game score of each intelligent strategy based on the Elo algorithm.

[0050] Step S3: Calculate the time-varying average game integrals of each intelligent strategy based on the moving average method.

[0051] Step S4: Based on the situational data of the adversarial process between various intelligent strategies, multiple comprehensive evaluation indicators are used for data statistics;

[0052] Step S5: Use a matrix to represent the win rates between each intelligent strategy, use a curve graph to represent the trend of the average game score of each intelligent strategy over time, use a bar chart to represent the final game score ranking of each intelligent strategy, and use a radar chart to represent the statistical results of each comprehensive evaluation index.

[0053] In some alternative implementations, step S1 further includes:

[0054] Step S11: Use a computing cluster to conduct large-scale adversarial tests in parallel. In the simulation environment, conduct one-on-one adversarial games between various intelligent strategies. According to the round-robin tournament system, under random initial conditions, any two intelligent strategies will play adversarial games for no less than a set number of rounds. The result of the win or loss between intelligent strategies is set to three situations: victory, defeat, and draw.

[0055] Step S12: Use a structured database to store the game process data and game results;

[0056] Step S13: Calculate the win rates between each intelligent strategy.

[0057] In some alternative implementations, in step S11, the number of sessions is set to 1000.

[0058] For example, four adversarial game strategies are selected for evaluation: Algorithm A, Algorithm B, Algorithm C, and the baseline opponent. In each round of the round-robin tournament, the game schedule between all participating strategies is arranged in a home-and-away double round-robin format, repeated 1000 times. Therefore, each round of the round-robin tournament involves 12000 adversarial trials, generating 12000 adversarial results (the fault tolerance mechanism ensures that all adversarial results are valid, and invalid matches are not recorded).

[0059] In some optional implementations, in step S13, when calculating the win rates between various intelligent strategies, the win rate of intelligent strategy A against intelligent strategy B is calculated using the following formula:

[0060]

[0061] In this embodiment, a win rate matrix is ​​used, and the win rate statistics are as follows:

[0062] Baseline opponent -- 8.95％ 17.60％ 2.75％ Algorithm A 91.05％ -- 41.20％ 12.00％ Algorithm B 82.40％ 58.80％ -- 51.45％ Algorithm C 97.25％ 88.00％ 49.55％ --

[0063] Here, -- indicates data with no adversarial results.

[0064] In some optional implementations, step S2, calculating the game integrals of each intelligent strategy based on the Elo algorithm, includes:

[0065]

[0066]

[0067] R' A =R A +K(S A -E A );

[0068] R' B =R B +K(S B -E B );

[0069] Among them: E A Let E be the expected win rate of intelligent strategy A. B Let R be the expected win rate of intelligent strategy B. A For the current integral of intelligent policy A, R B R' is the current integral of intelligent policy B.A R' is the integral after adversarial action against intelligent policy A. B Let S be the integral after adversarial action against intelligent policy B, where K is a constant coefficient and S is the integral. A With S B The value is the win / loss score of the intelligent strategy, with 1 for a win, 0.5 for a draw, and 0 for a loss.

[0070] In this embodiment, K is typically set to 16, and the initial R A R B The value can typically be 1200 or 1500. In alternative implementations, other values ​​can also be used.

[0071] In some alternative implementations, step S3 further includes:

[0072] Step S31: Record the game score of each intelligent strategy after each set number of confrontations.

[0073] Step S32: Take a window length of T = 10, and denote the game integral at time t as A. t Then, the average game integral F of intelligent strategy A is calculated using the moving average method. t When using this formula:

[0074]

[0075] Understandably, to avoid significant fluctuations in the game scores calculated by the Elo algorithm due to the recent match results, this embodiment uses a moving average method to smooth the game score sequence over time. In a round-robin tournament, if the four intelligent strategies compete against each other, with any two strategies playing 1000 matches each (home and away), the total number of matches is 12000. Game scores are recorded every 100 matches, resulting in a sequence of game scores over time. The final calculation results of the game scores for each intelligent strategy using the above method based on the match data are as follows:

[0076] Baseline opponent: 1080.91 points;

[0077] Algorithm A: 1409.47 points;

[0078] Algorithm B: 1563.92 points;

[0079] Algorithm C: 1546.11 points.

[0080] In some optional implementations, step S4 involves statistical analysis of data based on situational data from the adversarial process between various intelligent strategies, using multiple comprehensive evaluation indicators, including:

[0081] Informative indicators are determined based on radar scan tracking time data between various intelligent strategies:

[0082]

[0083] Energy indicators are determined based on the integral data of kinetic and potential energy over time for each intelligent strategy:

[0084]

[0085] Offensive metrics are determined based on the number of times each intelligent strategy knocks down opponents in winning matches:

[0086]

[0087] Security metrics are determined based on the number of matches in which each intelligent strategy is defeated by the opponent.

[0088]

[0089] Robustness metrics are determined based on the ratio of the lowest to the highest win rates of each intelligent strategy against other intelligent strategies.

[0090]

[0091] In this example, based on the outcome and process data of the game, the comprehensive evaluation index of each intelligent strategy is calculated using the above method, and the results are as follows:

[0092] Baseline opponent: Informationality 0.62, Energy 0.34, Offensiveness 0.17, Security 0.11, Robustness 0.78;

[0093] Algorithm A: Informativeness 0.52, Energy 0.61, Attack Power 0.49, Security 0.36, Robustness 0.11;

[0094] Algorithm B: Informativeness 0.45, Energy 0.30, Attack Power 0.25, Security 0.29, Robustness 0.48;

[0095] Algorithm C: Informativeness 0.47, Energy 0.54, Attack Power 0.24, Security 0.42, Robustness 0.36.

[0096] In some optional implementations, in step S5, the informational index, energy index, strikeability index, security index, and robustness index are normalized to between 0 and 1, and radar charts are drawn using the five axes corresponding to the above indexes (informational, energy, strikeability, security, and robustness), and then superimposed together for comparison and display.

[0097] These five evaluation metrics are all normalized to a range of 0 to 1. A value closer to 1 indicates that the corresponding intelligent strategy performs better under that metric.

[0098] In step S5, data visualization methods are used to compare and display the advantages and disadvantages of the intelligent strategies. The specific data visualization methods are as follows:

[0099] (5a) Win rate visualization uses a table to show the name and win rate of each strategy, and also uses a table to show the mutual win rate matrix between different strategies.

[0100] (5b) The points ranking visualization uses tables and bar charts to show the name and points of each strategy;

[0101] (5c) The comprehensive evaluation visualization adopts the form of radar chart, normalizes the evaluation indicators to between 0 and 1, and uses information, energy, strikeability, security and robustness as 5 axes. Each strategy draws its corresponding radar chart and they are superimposed together for comparison and display.

[0102] (5d) The visualization of score change trends uses a line graph to show the average game score change curve of each strategy during the round-robin process.

[0103] In some alternative implementations, step S5 further includes a visualization analysis of the reasons for failure, which includes displaying statistics on the reasons for failure (retreat or destruction) for each strategy during the round-robin tournament using a dynamically scrolling bar chart.

[0104] In some alternative implementations, after step S5, the following is further included:

[0105] Based on the collected adversarial results between various intelligent strategies, the total win rate of each intelligent strategy relative to other intelligent strategies is calculated and normalized to obtain the first evaluation parameter.

[0106] The game integrals of each intelligent strategy are normalized to obtain the second evaluation parameter;

[0107] The first evaluation parameter and the second evaluation parameter are weighted using a weighted average method to obtain the total score of each intelligent strategy.

[0108] By combining win rate and score in the above way, the strengths and weaknesses of intelligent strategies can be described more accurately.

[0109] This application enables a scientific and quantitative analysis of the advantages and disadvantages of adversarial intelligent strategies, providing data support for the improvement of intelligent strategies.

[0110] Although this application has been described in detail above with general descriptions and specific embodiments, some modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, such modifications or improvements made without departing from the spirit of this application are all within the scope of protection claimed in this application.

Claims

1. A method for evaluating intelligent strategies in adversarial games, characterized in that, include: Step S1: Based on the collected adversarial results between various intelligent strategies, calculate the mutual win rate between various intelligent strategies; Step S2: After each confrontation between the various intelligent strategies, calculate the game score of each intelligent strategy based on the Elo algorithm. Step S3: Calculate the time-varying average game integrals of each intelligent strategy based on the moving average method. Step S4: Based on the situational data of the adversarial process between various intelligent strategies, perform data statistics using comprehensive evaluation indicators, including: Informative indicators are determined based on radar scan tracking time data between various intelligent strategies: ； Energy indicators are determined based on the integral data of kinetic and potential energy over time for each intelligent strategy: ； Offensive metrics are determined based on the number of times each intelligent strategy knocks down opponents in winning matches: ； Security metrics are determined based on the number of matches in which each intelligent strategy is defeated by the opponent. ； Robustness metrics are determined based on the ratio of the lowest to the highest win rates of each intelligent strategy against other intelligent strategies. ； Step S5: Use a matrix to represent the win rates between each intelligent strategy, use a curve graph to represent the trend of the average game score of each intelligent strategy over time, use a bar chart to represent the final game score ranking of each intelligent strategy, and use a radar chart to represent the statistical results of each comprehensive evaluation index. In step S5, the informational index, energy index, strikeability index, security index, and robustness index are normalized to between 0 and 1, and radar charts are drawn using the five axes corresponding to the above indexes: informational, energy, strikeability, security, and robustness. These charts are then superimposed and compared for display. Following step S5, the following further includes: Based on the collected adversarial results between various intelligent strategies, the total win rate of each intelligent strategy relative to other intelligent strategies is calculated and normalized to obtain the first evaluation parameter. The game integrals of each intelligent strategy are normalized to obtain the second evaluation parameter; The first evaluation parameter and the second evaluation parameter are weighted using a weighted average method to obtain the total score of each intelligent strategy.

2. The intelligent strategy evaluation method for adversarial games as described in claim 1, characterized in that, Step S1 further includes: Step S11: Use a computing cluster to conduct large-scale adversarial tests in parallel. In the simulation environment, conduct one-on-one adversarial games between various intelligent strategies. According to the round-robin tournament system, under random initial conditions, any two intelligent strategies will play adversarial games for no less than a set number of rounds. The result of the win or loss between intelligent strategies is set to three situations: victory, defeat, and draw. Step S12: Use a structured database to store the game process data and game results; Step S13: Calculate the win rates between each intelligent strategy.

3. The intelligent strategy evaluation method for adversarial games as described in claim 2, characterized in that, In step S11, the number of sessions is set to 1000.

4. The intelligent strategy evaluation method for adversarial games as described in claim 2, characterized in that, In step S13, when calculating the win rates between various intelligent strategies, the win rate of intelligent strategy A against intelligent strategy B is calculated using the following formula: 。 5. The intelligent strategy evaluation method for adversarial games as described in claim 1, characterized in that, In step S2, the game integrals of each intelligent strategy are calculated based on the Elo algorithm, including: ； ； ； ； Where: EA is the expected win rate of intelligent strategy A, EB is the expected win rate of intelligent strategy B, RA is the current score of intelligent strategy A, RB is the current score of intelligent strategy B, R'A is the score of intelligent strategy A after the confrontation, R'B is the score of intelligent strategy B after the confrontation, K is a constant coefficient, and SA and SB are the result win / loss scores of intelligent strategies, with 1 for a win, 0.5 for a draw, and 0 for a loss.

6. The intelligent strategy evaluation method for adversarial games as described in claim 1, characterized in that, Step S3 further includes: Step S31: Record the game score of each intelligent strategy after each set number of confrontations. Step S32: Take a window length of T=10, and denote the game integral at time t as... When calculating the average game integral Ft of intelligent strategy A using the moving average method, the following formula is used: 。 7. The intelligent strategy evaluation method for adversarial games as described in claim 1, characterized in that, Step S5 further includes a visualization analysis of the reasons for failure, which includes using a dynamically scrolling bar chart to display statistics on the reasons for failure of each strategy during the round-robin tournament.

Images