Layered adversarial reasoning method

Abstract

The application discloses a hierarchical confrontation deduction method for training a virtual AI team with a target team style. The hierarchical confrontation deduction method comprises the following steps: obtaining event data and tracking data of a real football match; calculating tactical indexes of the target team in different match phases and counting and aggregating to obtain a style constraint set; establishing a macroscopic tactical strategy mapping function to generate a tactical instruction according to a match state and the style constraint set; extracting a macroscopic match state in a virtual football simulation environment and generating a tactical instruction through the macroscopic tactical strategy mapping function; each execution subject selects and executes an individual action through an individual action strategy, and updates a strategy parameter through reinforcement learning based on trajectory data; and iterative optimization is performed until a training termination condition is met. The application realizes effective guidance of real data to virtual reinforcement learning, improves training stability and maintains consistency of a tactical style.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and sports analytics, specifically to a method for abstracting team tactical styles based on real football match data and guiding reinforcement learning training in a virtual football simulation environment through layered adversarial simulation. Background Technology

[0002] With the development of artificial intelligence technology, data-driven methods are increasingly being applied to fields such as sports match analysis, tactical evaluation, and training assistance. In football, the acquisition of large-scale match event data and player tracking data makes it possible to analyze and quantify the overall tactical behavior of teams and player decisions. Building intelligent analysis models based on large-scale, high-quality match data to simulate or optimize team performance has become one of the important research directions in sports performance analysis and data analysis.

[0003] In existing technologies, methods for modeling decision-making in football matches mainly include rule-based tactical systems, behavior analysis methods based on statistical learning, and simulation training methods based on reinforcement learning or imitation learning. Reinforcement learning methods typically optimize the action strategies of one or more decision-makers through repeated trial and error in a virtual football simulation environment; imitation learning methods attempt to directly learn the behavioral patterns of players or teams using historical match data. However, most of these methods focus on learning atomic actions or isolated individual actions, or indirectly induce specific behavioral characteristics through reward functions, making it difficult to explicitly model and control the overall tactical style exhibited by a team in different phases of the match.

[0004] Furthermore, there are often significant differences between real football match data and virtual simulation environments in terms of state representation, action space, and environmental dynamics. In existing technologies, real data is often used to train policy models or forcibly fit and align parameters with the simulation environment, which to some extent limits the model's generalization ability and training stability. Meanwhile, a general, structured, and scalable technical solution remains lacking for effectively utilizing the macro-tactical patterns inherent in real match data to guide the learning process in the virtual environment while maintaining the online adaptability of reinforcement learning. Summary of the Invention

[0005] This disclosure relates to the fields of artificial intelligence and sports analytics, specifically to a deductive method for abstracting team tactical styles based on real football match data and guiding reinforcement learning training in a virtual football simulation environment through layered adversarial training.

[0006] This disclosure provides a layered adversarial deduction method for training a virtual AI team with the style of a target team. The deduction method includes: Step 1: acquiring real football match data of the target team, including text-based event data and tracking data; Step 2: based on the real football match data, calculating the tactical indicators of the target team under different match phases, and statistically aggregating the tactical indicators from multiple historical matches to obtain a style constraint set, wherein the match phase refers to the tactical stage divided according to the possession status and tactical objectives, and the tactical indicators are used to quantitatively describe the tactical behavioral characteristics of the team; Step 3: establishing a macro-tactical strategy mapping function. Step 4: In the virtual football simulation environment, extract the macro-level match state from the match state of the virtual football simulation environment. Based on the macro-level match state and the set of style constraints, generate tactical instructions describing the team's overall tactical intentions in the current match phase. The match state includes motion state information representing the motion characteristics of the participants in the match and context information representing the match process. The participants' execution entities and their interactions, along with the macro-level match state, are derived from the aggregation of the execution entities' position, speed, and ball possession information, used to characterize the team's overall formation structure, ball possession status, and spatial distribution. Step 5 involves each execution entity independently selecting and executing individual actions through individual action strategies. These individual action strategies are policy functions that allow the execution entity to select individual actions based on tactical instructions and local states. Execution entities interact with the virtual football simulation environment by executing individual actions, obtaining trajectory data. Based on this trajectory data, the parameters of the macro-level tactical strategy mapping function are updated using reinforcement learning methods. The parameters of the individual action strategy are as follows: the local state is a subset of the game state, including the position, speed, and ball possession information of the executing subject and other executing subjects within the predetermined range of the executing subject; the parameter update of the macro tactical strategy mapping function is optimized based on the degree of matching between the generated tactical instructions and the set of style constraints; the parameter update of the individual action strategy is optimized based on the consistency between the game task objective and the tactical instructions. Step 6: Iteratively execute steps 4 and 5 to optimize the parameters of the macro tactical strategy mapping function and the parameters of the individual action strategy until the preset training termination condition is met, and output the parameters of the iteratively updated macro tactical strategy mapping function and the parameters of the individual action strategy.

[0007] According to one or more embodiments of this disclosure, the step of initializing the parameters of the macro-tactical strategy mapping function includes: initializing the parameters of the macro-tactical strategy mapping function based on a set of style constraints; or using tactical decision samples from real match data to perform supervised pre-training on the parameters of the macro-tactical strategy mapping function to initialize the parameters of the macro-tactical strategy mapping function; or using a random initialization method to initialize the parameters of the macro-tactical strategy mapping function.

[0008] According to one or more embodiments of this disclosure, a method for extracting macroscopic match state from the match state of a virtual football simulation environment includes: extracting macroscopic features reflecting the team's overall formation structure, ball possession status, and space occupancy distribution from the environmental state containing the position, speed, and ball possession information of the executing entity.

[0009] According to one or more embodiments of this disclosure, tactical instructions are encoded as a set of structured macro-control parameters to describe the team's overall space utilization, offensive pace, defensive positioning, and direction of offensive and defensive transitions in the current game phase.

[0010] According to one or more embodiments of this disclosure, macroscopic control parameters are parameters for regulating the geometric topology of the overall spatial structure of the team. The macroscopic control parameters include: a center of gravity position parameter for defining the overall center of gravity coordinates of the team; a stretching-compression parameter for defining the dispersion or compactness of the distribution of team players; and a parameter for defining the distribution weight of the team in the lateral space or the width utilization preference of the target area.

[0011] According to one or more embodiments of this disclosure, the objective function for optimizing the parameter update of the macro-tactical strategy mapping function is: ,in, The parameters of the macro-tactical strategy mapping function are... Expressing expectations, As a discount factor, This is a macro-tactical reward function used to measure the degree to which the generated tactical instructions match the set of style constraints in at least one of the following aspects: space utilization, advance pace, defensive positioning, and offensive-defensive transitions. Indicates the overall state of the game. This represents the tactical command at time t.

[0012] According to one or more embodiments of this disclosure, the objective function for optimizing the parameter updates of an individual action strategy is: ,in, For individual action strategy parameters, The degree to which the game objectives have been achieved, as reflected by the virtual football simulation environment based on the game rules. This is a consistency evaluation function used to measure the consistency between individual actions and tactical instructions. For weight parameters, This represents the individual action of the i-th executing entity at time t.

[0013] According to one or more embodiments of this disclosure, the preset training termination conditions include at least one of the following conditions: the number of training rounds or interaction steps reaches a preset upper limit; the performance evaluation index in the virtual competition changes less than a preset threshold over multiple consecutive training cycles; the macro-tactical reward function converges or tends to stabilize; and the consistency evaluation function converges or tends to stabilize.

[0014] According to one or more embodiments of this disclosure, event data includes one or more of passing, shooting, and interception, and tracking data includes the spatial coordinates, velocity vectors, ball position, and possession status of each player at each moment.

[0015] According to one or more embodiments of this disclosure, the style constraint set can be represented as a probability distribution, a weight vector, a threshold rule, or any combination thereof.

[0016] According to one or more embodiments of this disclosure, the game phase includes one or more of a positional attack phase, a defensive phase, and a transition phase, wherein the transition phase includes an attack-to-defense phase and a defense-to-attack phase.

[0017] According to one or more embodiments of this disclosure, tactical metrics include one or more of the following: progressive action rate, half-space occupation rate, and packing rate in the attacking phase; and / or one or more of the following of the following of the following of the defensive phase: pressing intensity and defensive line height; and / or one or more of the following of the following of the following of the transition phase: rebuilding time, counter-pressing success rate, average speed of team retreat, and cumulative expected goals of the opponent.

[0018] This disclosure also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program; wherein the processor executes the computer program to implement the deduction method described above.

[0019] The hierarchical adversarial deduction method provided in this disclosure introduces team styles embedded in real match data as an intermediate strategy layer into a hierarchical reinforcement learning framework within a virtual simulation environment. This enables offline real data to effectively guide the reinforcement learning process of multiple online agents, significantly reducing the exploration space and improving training stability while ensuring policy adaptability. Furthermore, it ensures that the trained virtual team maintains consistency with the target team's style in overall tactical behavior, providing a reliable basis and specific method for team tactical deduction and verification. The method disclosed has good versatility and scalability, applicable to different teams, leagues, and virtual simulation environments, and can be applied to various multi-agent decision support scenarios such as team sports tactical analysis. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0021] Figure 1 This is a flowchart of a layered adversarial deduction method based on macro-strategy abstraction according to an embodiment of this application.

[0022] Figure 2 This is a structural block diagram of a hierarchical adversarial inference method based on macro-strategy abstraction according to an embodiment of this application.

[0023] Figure 3 This is a block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such order can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0027] The present invention will now be described in detail with reference to specific embodiments.

[0028] Figure 1 This is a flowchart of a layered adversarial deduction method according to an embodiment of the present disclosure. Figure 2 This is a structural block diagram of the deduction method for layered confrontation according to the embodiments of this disclosure.

[0029] Reference Figure 1 and Figure 2 This invention provides a method for deducing hierarchical adversarial strategies, used to train a virtual AI team strategy model with the tactical style of a target team in a virtual football simulation environment.

[0030] In step 1, real football match data of the target team is obtained. This real football match data includes text-based event data and tracking data.

[0031] In this embodiment, event data includes one or more of passing, shooting, and interception, and tracking data includes the spatial coordinates, velocity vectors, ball position, and possession status of each player at each moment.

[0032] Real football match data can be historical data of the target team in actual matches. The match event data can be obtained by annotating match videos using professional analysis software. Tracking data can be extracted from match videos using computer vision technologies, or collected by wearing GPS or UWB related devices. The acquired event data records key events that occur in sequence and at irregular intervals during the match, such as passing, receiving, interception, and shooting. Each event includes relevant information such as the players involved, their positions, the location where the event occurred, and the location where it ended. The level of detail depends on the standards used to record the events. The acquired tracking data records the position information of all 22 players and the ball at every moment during the match. The positions can be absolute GPS information or relative coordinates, i.e., establishing a 2D or 3D coordinate system around a point on the football field, and recording data at fixed intervals of 10Hz to 25Hz.

[0033] During the data preprocessing stage, real football match data can undergo operations such as timestamp alignment, spatial coordinate unification, and missing value handling to form standardized input data for tactical analysis. Specifically, starting with the kickoff in the event data, the timestamps of both are unified; if the coordinate systems are different, one is used as the standard to transform and calculate the other's spatial coordinates; missing values ​​and other data anomalies are handled according to standard data processing procedures.

[0034] In step 2, based on real football match data, tactical indicators for the target team under different match phases are calculated, and the tactical indicators from multiple historical matches are statistically aggregated to obtain a set of style constraints. Here, match phase refers to the tactical stage divided according to the state of possession and tactical objectives, and tactical indicators are used to quantitatively describe the team's tactical behavioral characteristics.

[0035] In the embodiments, the style constraint set is represented in the form of a probability distribution, a weight vector, a threshold rule, or any combination thereof.

[0036] In the embodiments, the game phase includes one or more of the following: offensive phase (also known as offensive segment or offensive phase), defensive phase (also known as defensive segment or defensive phase), and transition phase (also known as transition segment or offensive-defense transition phase), wherein the transition phase includes the offensive-to-defensive phase (also known as the offensive-to-defensive phase) and the defensive-to-offensive phase (also known as the defensive-to-offensive phase).

[0037] In the embodiments, tactical indicators include one or more of the following: progressive action rate, half-space occupation rate, and packing rate in the attacking phase; and / or one or more of the following of the following of the following in the defensive phase: pressing intensity and defensive line height; and / or one or more of the following of the following in the transition phase: rebuilding time, counter-pressing success rate, average speed of team retreat, and cumulative opponent expected goals (also known as opponent xG cumulative).

[0038] More specifically, match phases refer to the different tactical stages into which a football match is divided based on possession, match status, and tactical objectives. In the field of football tactical analysis, match phase division is a widely used analytical framework for decomposing a continuous match into discrete stages with different tactical characteristics. Teams exhibit significant differences in tactical objectives, organizational methods, and behavioral patterns across different phases, thus requiring separate tactical style modeling for each phase.

[0039] Based on event and tracking data, the game process is divided into phases, and the team's overall tactics are broken down into tactical behaviors under different game phases. A game phase includes at least an offensive phase, a defensive phase, and a transition phase. The division of game phases can be done through quantifying expert knowledge into rules, or through model learning by training a model using labeled partial division results, thus enabling the division of a large number of games. Depending on specific needs, different tactical phases are defined for each phase; for example, the offensive phase includes positional attacks and backcourt organization, while the transition phase includes offense-to-defense and defense-to-attack.

[0040] In the field of football tactical analysis, the division of the game phase is mainly based on the following judgment criteria: (1) Possession status: by analyzing events such as passing, losing possession, and interception in the event data, it is determined which side has possession at the current moment; (2) Team formation and position: by using tracking data to calculate the team's center of gravity position, compression degree, and other characteristics, it is determined whether the team is in a high-pressing or low-defense position; (3) Game rhythm characteristics: by analyzing the time interval of consecutive events and the player's movement speed, different rhythm patterns such as fast counterattacks or positional organization are identified. In specific implementation, a rule-based discriminant function can be constructed. ,in This indicates the ball possession status at time t. This indicates the team's focus and formation characteristics. These represent the parameters of the game's tempo. When these features meet specific combination conditions, they are determined to be the corresponding game phase.

[0041] In this embodiment, for different tactical stages under each phase, statistical analysis or model analysis is performed on the team's overall tactical behavior to abstract a set of constraints (also known as the style constraint set) representing the team's macro-tactical style. The style constraint set describes the overall preference characteristics of different teams in terms of space utilization, offensive organization, defensive methods, and transitions between offense and defense. The style constraint set can be obtained from real football match data through various methods such as statistical analysis, machine learning models, and expert knowledge definitions.

[0042] It should be noted that the relationship between the style constraint set and tactical indicators is as follows: tactical indicators are intermediate calculation results quantifying tactical behavior in a single game or at a single moment, while the style constraint set is a statistical aggregation of tactical indicator values ​​across multiple games, used to describe the typical style range and preference distribution of a team. Specifically, by calculating the values ​​of each tactical indicator for multiple historical games of the target team, a value sequence for each indicator is obtained. Then, statistical analysis is performed on this sequence (such as calculating the mean, standard deviation, quantiles, etc.), and the statistical results are expressed as probability distributions, value ranges, or threshold rules, thus forming the style constraint set for that phase. Therefore, the style constraint set is an aggregated abstraction of tactical indicators in the time and sample dimensions, reflecting the stability characteristics of a team's tactical behavior rather than a single instance of performance.

[0043] In this embodiment, tactical indicators refer to calculable metrics used to quantify the characteristics of a team's tactical behavior. A team's tactical objectives differ across different phases of the game, thus requiring the selection of different tactical indicators: in the offensive phase, the focus is on how the team advances, organizes, and creates opportunities, therefore indicators such as progressive action rate and space utilization rate are used; in the defensive phase, the focus is on how the team presses, intercepts, and protects space, therefore indicators such as pressing intensity and defensive line height are used; in the transition phase, the focus is on the speed and efficiency of the team's offensive and defensive transitions, therefore indicators such as rebuilding time and counter-pressing success rate are used. The selection of indicators for each phase follows the core tactical task of that phase.

[0044] The methods for determining tactical indicators can be based on the understanding of the core tactical elements of each phase by coaches and tactical analysis experts, selecting quantifiable indicators that can reflect these elements. Alternatively, statistical methods such as cluster analysis and principal component analysis can be used to automatically discover key characteristic dimensions that can distinguish different team styles from a large amount of match data. Furthermore, correlation analysis or causal inference methods can be used to identify tactical behavioral characteristics that are significantly related to match results (such as the number of goals, win rate, etc.) as key indicators.

[0045] The set of style constraints can be described as a set of tactical indicators, including but not limited to macro-behavioral attributes such as the team's spatial occupation characteristics, the relative positional relationships of players, the offensive advancement methods, the intensity of defensive pressure, and the rhythm of offensive and defensive transitions. By conducting statistical analysis or model learning on a large number of historical samples, the distribution characteristics of different indicators at each tactical stage are calculated. This is not a limitation on a single action or individual player behavior, but rather an abstract expression of the structural preferences of the team's overall tactical behavior.

[0046] As an example, taking the target team as the benchmark, dividing the game into multiple phases such as offense, defense, and transition can be represented as follows: There are m phases in total, and a state representation is constructed for each phase to characterize the team's overall tactical behavior: ,in, This represents the spatial position of the i-th player at time t. This represents the corresponding motion state characteristics. Indicates ownership of the ball. Let t represent the spatial position of the ball at time t, and p represent the current tactical phase. For example, but not limited to, in the positional attack phase under the attacking phase, three tactical indicators can be abstracted: progressive actions rate, half-space occupancy rate, and packing rate. In the transition phase from attack to defense, four tactical indicators can be abstracted: recovery time, counter-press success rate, average retreat speed of the whole team, and cumulative turnover concession xG. Each indicator can be calculated using specific formulas or models derived from a large amount of historical match data through statistical analysis or model learning. In the embodiment, progressive actions rate refers to the frequency with which the team significantly advances the ball closer to the opponent's goal area through passing or dribbling. Half-space occupancy rate refers to the proportion of player activity in the high-value area of ​​the "flank" (the most threatening tactical corridor) between the center and the flanks of the field. Packing rate refers to the number of opposing defensive players successfully bypassed and rendered ineffective by a single pass or dribble, used to measure the penetration of the attack. Rebuilding time refers to the time it takes for a team to reorganize and solidify its defensive formation from the moment it loses possession of the ball.

[0047] More specifically, taking the gradual action rate index in the positional attack phase as an example, this illustrates how tactical indicators can be calculated from the event data and tracking data obtained in step 1.

[0048] Extract all passing, dribbling, and shooting events from the event data obtained in step 1, and denot them as the event sequence. Each event Includes timestamp Implementation entity (players) Starting position ( ), End position ( and event type .

[0049] When determining progressive behavior, for each event To determine whether an event constitutes "progressive behavior," the following rule applies: if an event brings the ball closer to the opponent's goal and the longitudinal distance of the advance exceeds a threshold (e.g., 10 meters), then the event is considered "progressive behavior." Miqie (Assuming the field length is 105 meters and the opponent's goal is located at y=105), this event is marked as an asymptotic behavior.

[0050] In this embodiment, only progressive actions occurring within the offensive phase are counted. Based on the aforementioned phase division results, it is determined whether each time period belongs to the offensive phase, and only events within the time periods belonging to the offensive phase are counted.

[0051] The asymptotic behavior rate is defined as the frequency of asymptotic behavior occurring per unit time, and the calculation formula is: ,in, The total number of progressive advances identified within the phase of the positional attack is defined as follows. The total possession time (in minutes) of the team during positional attack phases is obtained by summing up the possession information from event data and tracking data.

[0052] Calculate the PAR value for multiple historical matches of the target team (e.g., the last 10 matches) to obtain a sequence. Take their average and standard deviation As a stylistic constraint on the team's gradual offensive behavior during positional attacks, it can be expressed as: The probability distribution form of .

[0053] Suppose that in a match, a team's positional attack phase lasts for a total of 18 minutes. During this period, 54 passes and dribbling actions that met the progressive standards were completed. If the PAR values ​​for the team in the 10 matches are {2.8, 3.2, 3.0, 2.9, 3.1, 3.3, 2.7, 3.0, 3.2, 2.9}, then their style constraint is... This indicates that the team executes approximately 3 progressive actions per minute on average during positional attacks, with a deviation of approximately 0.18 actions per minute.

[0054] Similarly, other tactical metrics such as half-space utilization, packing rate, and rebuild time can be extracted and calculated from event and tracking data by defining corresponding judgment rules and calculation formulas. Different metrics focus on different aspects of tactical behavior, and the combination of multiple metrics can comprehensively characterize the team's tactical style under specific phases.

[0055] Through statistical or model analysis of a large number of historical samples at various phases, typical style constraint indicators describing the target team at each phase are obtained and mapped to high-level strategies (also known as macro-tactical strategies), which can be expressed as follows: ,in This represents the set of style constraints under phase p, where each variable... This refers to a specific stylistic constraint on a team, used to define the team's overall preferences in space utilization, offensive organization, defensive coordination, and transition strategies. For example, but not limited to, during positional attacks... The set of style constraints is as follows The superscript oo indicates the organized offense phase, and the subscripts PAR, HsOR, and Pack correspond to three tactical indicators: progressive action rate, half-space occupancy rate, and packing rate, respectively. , , These represent style constraints derived from gradual behavior rate, half-space occupancy rate, and packing rate during the positional attack phase, respectively.

[0056] In step 3, a macro-tactical strategy mapping function is established and its parameters are initialized. This function generates tactical instructions describing the team's overall tactical intent in the current match phase, based on the match state and a set of style constraints. The match state includes motion state information characterizing the movement characteristics of the participants and contextual information characterizing the match's progress.

[0057] In this embodiment, the step of initializing the parameters of the macro-tactical strategy mapping function includes: initializing the parameters of the macro-tactical strategy mapping function based on the style constraint set; or using tactical decision samples from real match data to perform supervised pre-training on the parameters of the macro-tactical strategy mapping function to initialize the parameters of the macro-tactical strategy mapping function; or using a random initialization method to initialize the parameters of the macro-tactical strategy mapping function.

[0058] Based on the style constraint set obtained in step 2, construct and initialize the macro-tactical strategy mapping function so that it can output tactical instructions that conform to the style of the target team according to the current game status.

[0059] The macro-tactical strategy mapping function (also known as high-level strategy) refers to a decision function that generates tactical instructions based on the constraints of the team's overall tactical style and the current match status. It forms a hierarchical decision structure with multiple individual action strategies (also known as low-level decision units or low-level agents). The macro-tactical strategy mapping function is used to output tactical instructions, and the multiple individual action strategies are used to execute specific actions. The multiple individual action strategies correspond to different execution subjects (such as players in different positions) in the virtual football simulation environment.

[0060] Specifically, the macro-tactical strategy mapping function can be formally expressed as: ,in This represents the macroscopic game state (which will be extracted from simulation environment observations in step 4). Let p represent the set of style constraints under phase p (obtained from step 2). This represents the output tactical instructions. The macro-tactical strategy mapping function can be implemented through a parameterized model, denoted as... ,in The parameters of the macro-tactical strategy mapping function (also known as high-level strategy parameters) need to be initialized in this step and optimized and updated during subsequent training.

[0061] In the embodiment, the parameters of the macro-tactical strategy mapping function The initialization process may include: converting each constraint obtained in step 2... Transform statistical distributions into conditional rules that can be executed within a policy network. For example, convert the packing rate constraint under a positional attack phase. Converted into executable action selection rules: When making passing or dribbling decisions, prioritize action paths that can bypass multiple opposing players, i.e., satisfy... The action, after transformation, can be represented as: In this context, i, j, and k represent the player's ID number (i.e., the entity that performs the offensive action); move, carry, and pass represent the three types of actions performed by the entity: movement, dribbling, and passing, respectively. This represents the coordinates of the ball along the direction of attack (i.e., the longitudinal direction towards the opponent's goal) at the moment the action begins. This indicates the coordinates of the ball along the direction of attack at the moment the action ends. This represents the displacement of the ball along the attacking direction caused by the action; d represents the displacement threshold, which can be determined based on the mean and variance of the constraint distribution. This rule states that when the actors i, j, and k perform movement, dribbling, or passing actions, the displacement of the ball along the attacking direction must be greater than the threshold d, thereby guiding behavior at the micro level that conforms to the macro-level packing rate style.

[0062] Then, based on the dimensions and types of the style constraint set, the input-output structure of the macro-tactical strategy network is designed. The input layer receives the macro-game state. Feature vectors, intermediate layer coding style constraints Impact on tactical decision-making; the output layer generates tactical instructions. The various components (such as spatial compression, propulsion rhythm, width utilization, etc.).

[0063] Random initialization, expert knowledge-based initialization, or pre-training methods can be used to initialize... Initialization can be performed. For example, supervised pre-training can be conducted using tactical decision samples from real match data, enabling the initial strategy to generate tactical instructions that closely resemble the target team's style; or it can be based on a set of style constraints. The statistical characteristics (such as mean and variance) in the network are used to set the bias parameters of the output layer, so that the initial output distribution matches the constraint distribution.

[0064] Through the above initialization process, the parameters of the initial macroscopic tactical strategy mapping function are obtained. This serves as the starting point for subsequent reinforcement learning training. The initialization strategy can be based on the style constraint set. Generate tactical instructions that align with the target team's style of play, providing effective guidance for training in a virtual simulation environment.

[0065] In step 4, within the virtual football simulation environment, a macro-level match state is extracted from the match state of the virtual football simulation environment. Based on the macro-level match state and the style constraint set, the tactical instructions are generated through the macro-level tactical strategy mapping function. The virtual football simulation environment includes multiple independent action decision-makers acting as participants in the virtual match, as well as interactions between these entities. The macro-level match state is obtained by aggregating the position, speed, and ball possession information of the entities and is used to characterize the team's overall formation structure, ball possession status, and spatial distribution.

[0066] In this embodiment, the step of extracting the macroscopic match state from the match state of the virtual football simulation environment includes: extracting macroscopic features reflecting the team's overall formation structure, ball possession status, and spatial occupancy distribution from the environmental state containing the position, speed, and ball possession information of the executing entity. As an example, spatial occupancy distribution refers to the areas of the field where players primarily concentrate their activities during a specific period of the match. This can be represented using data methods such as heatmaps or grid density.

[0067] In this embodiment, the tactical instructions are encoded as a set of structured macro-control parameters to describe the team's overall space utilization, offensive pace, defensive positioning, and direction of offensive and defensive transitions in the current game phase.

[0068] More specifically, macro-control parameters are parameters that regulate the geometric topology of the team's overall spatial structure, including: center of gravity position parameters used to define the team's overall center of gravity coordinates; stretch-compression parameters used to define the dispersion or compactness of the team's player distribution; and parameters used to define the team's distribution weight in the lateral space or the width utilization preference of the target area.

[0069] As a specific embodiment, the current match status information is obtained in the virtual football simulation environment, and the macro tactical strategy mapping function generates tactical instruction output based on the match status and style constraint set.

[0070] The virtual football simulation environment provides environmental state observations including player position, speed, movement status, ball possession information, and match context. The match state (also known as state or environmental state) can be formally represented as follows: It can be further mapped to a macroscopic game state representation used by the macroscopic tactical strategy mapping function through feature extraction or aggregation mapping. ,in This is a state abstraction function used to extract macroscopic features reflecting the overall formation structure, space distribution, ball possession, and offensive and defensive situations. Based on this, the macroscopic tactical strategy mapping function takes the macroscopic match state representation and a pre-constructed set of team tactical style constraints as input to generate corresponding tactical decision outputs.

[0071] In this embodiment, the macro-tactical strategy mapping function has been defined in step 3 as follows: Its parameterized form is ,in This refers to the set of macro-tactical style constraints for the team obtained from the aforementioned steps. This represents the tactical instructions or macroscopic control information (also known as macroscopic control parameters) output by the macroscopic tactical strategy mapping function at time t. These are the parameters for the macro-tactical strategy mapping function initialized in step 3. The tactical instructions describe the team's overall tactical intentions at the current stage, such as offensive advancement methods, defensive organization, spatial compression or stretching, and the rhythm of offensive and defensive transitions—these are macro-level behavioral directions, rather than directly corresponding to specific atomic actions (also known as individual actions). The macro-tactical strategy mapping function, through a comprehensive evaluation of the game's development trend and the set of tactical style constraints, ensures that the generated tactical decisions conform to the target team's true tactical preferences in terms of overall behavioral distribution, while also being dynamically adjusted according to changes in the game phase. Therefore, the tactical instructions output by the macro-tactical strategy mapping function serve as intermediate-level decision results, which are passed to the lower-level execution strategy (also known as individual action strategy, bottom-level decision unit) to guide the subsequent specific action generation process, thereby achieving a team-wide decision-making mechanism in the virtual simulation environment that balances real-time adaptability and style consistency.

[0072] Specifically, the virtual football simulation environment provides environmental state observations including the position, speed, motion state, ball possession information, and match context of the executing entity (also known as the player). The match state can be formally represented as... ,in This represents the spatial coordinate information of the i-th executing entity at time t. The corresponding velocity is represented as a vector. ,in Indicates the position of the ball. The status of possession of the ball is indicated by a Boolean value. This indicates that our team has possession of the ball. This indicates that the opponent has possession of the ball. Contextual information indicating the score and remaining time.

[0073] Constructing state abstract functions By mapping the real-time state of the game, a macro-level representation of the game state is obtained from the macro-level tactical strategy mapping function. The macro-level match state is used to characterize the team's overall formation structure, ball possession status, spatial distribution, and offensive and defensive situation; it is also combined with the set of style constraints of the tactical phase under the current phase p. Construct instruction generation function ,by and As input, tactical instructions are generated to guide the team's overall tactical behavior. .

[0074] In an embodiment, the tactical instructions Encoded as a set of structured macro-control parameters, the values ​​or ranges of which describe the team's overall space utilization, offensive pace, defensive positioning, and tactical intentions during the current tactical phase of the game. Specifically, this can be achieved by adjusting parameters that control the geometric topology of the team's overall spatial structure or by controlling the running range of relevant implementers. This can control the team's degree of stretching or compression in the lateral and longitudinal space, center of gravity position, and width utilization preferences; or by controlling the constraints on the movement space of relevant implementers during offensive advancement, thus controlling the team's longitudinal advancement, lateral transfer, or ball control organization.

[0075] In this way, the tactical instructions generated by the macro-tactical strategy mapping function do not directly limit the action decisions of individual executors, but rather impose structural constraints on the overall behavior of the team from a macro-level perspective of multi-person collaboration, thereby providing consistent tactical guidance for the subsequent action strategy selection of individuals in the virtual simulation environment.

[0076] In step 5, within the virtual football simulation environment, each executing entity independently selects and executes individual actions through an individual action strategy. The individual action strategy is a strategy function by which the executing entity selects the individual action to be executed based on tactical instructions and local states. The executing entity interacts with the virtual football simulation environment by executing individual actions to obtain trajectory data. Based on the trajectory data, the parameters of the macro-tactical strategy mapping function and the parameters of the individual action strategy are updated using reinforcement learning methods. The local states are a subset of the game states, including the position, speed, and ball possession information of the executing entity and other executing entities within a predetermined range. The parameter update of the macro-tactical strategy mapping function aims to optimize the matching degree between the generated tactical instructions and the set of style constraints, while the parameter update of the individual action strategy aims to optimize the consistency between the game task objective and the tactical instructions.

[0077] In this embodiment, the objective function for optimizing the parameter update of the macro-tactical strategy mapping function is: ,in, The parameters of the macro-tactical strategy mapping function are... Expressing expectations, As a discount factor, This is a macro-tactical reward function used to measure the degree to which the generated tactical instructions match the set of style constraints in at least one of the following aspects: space utilization, advance pace, defensive positioning, and offensive-defensive transitions. Indicates the overall state of the game. This represents the tactical command at time t.

[0078] In this embodiment, the objective function for optimizing the parameter updates of the individual action strategy is: ,in, For individual action strategy parameters, The degree to which the game objectives have been achieved, as fed back by the virtual football simulation environment according to the game rules. This is a consistency evaluation function used to measure the consistency between individual actions and tactical instructions. For weight parameters, This represents the individual action of the i-th executing entity at time t.

[0079] More specifically, tactical instructions are used to constrain or guide the action selection process of individual action strategies in a virtual simulation environment, ensuring that the individual action strategies maintain consistency with the tactical instructions while achieving the competition objective. The individual action strategies are trained using reinforcement learning methods.

[0080] Individual action strategies are trained using reinforcement learning, simultaneously considering the completion rate of task objectives (also known as match objectives, match objective tasks, or match task goals) and the degree of obedience to tactical instructions during training. Tactical instructions apply soft constraints to the action selection process of the individual action strategies through reward mechanisms, action probability weighting, policy regularization, or combinations thereof. The style constraint set is obtained based on offline real match data, while reinforcement learning training is conducted online within a virtual football simulation environment.

[0081] More specifically, each individual action strategy in the virtual simulation environment corresponds to an executor capable of making independent action decisions, and its local state at time t can be represented as follows: ,in This represents the individual state feature mapping function that extracts the individual's action strategy related to the i-th individual from global observation information (also known as environmental state observation). Each individual's action strategy outputs tactical commands based on its local state and the macroscopic tactical strategy mapping function. This generates the corresponding action selection, whose policy function can be expressed as: ,here This represents the atomic action (also called the individual action) chosen by the i-th individual action strategy at time t. The tactical instructions exert constraints or guidance on the decision-making process of the individual action strategy by regulating the action space, action preferences, or reward signals, and define an auxiliary evaluation function (also called a consistency evaluation function) to measure the consistency between the individual action strategy behavior and the tactical instructions. This allows for the application of soft constraints on action selection results, which are then incorporated into the reward calculation in reinforcement learning. This enables individuals to optimize their action strategies for the game objective while also considering their obedience to tactical instructions.

[0082] Individual action strategies are trained using reinforcement learning methods. Specifically, their objective function can be expressed as maximizing the expected cumulative reward over time. In this way, while individual action strategies cooperate to complete the competition objective, their overall behavior pattern gradually approaches the tactical intent described by the macro tactical strategy mapping function, thereby forming a team collaborative behavior pattern in the virtual simulation environment that combines task performance with tactical style consistency.

[0083] The above steps are repeatedly executed in the virtual simulation environment to update the policy parameters of the macro-tactical strategy mapping function and the individual action strategy until the preset training termination condition is met.

[0084] More specifically, based on the interaction trajectory data (also known as trajectory data or interaction trajectory) obtained in the virtual simulation environment, the parameters of the macro-tactical strategy mapping function and the parameters of the individual action strategy (also known as the underlying strategy parameters) are updated respectively to achieve consistent learning between macro-tactical decisions and micro-action execution. The interaction trajectory can be represented as... ,in This represents the macroscopic match status obtained in step 4. This represents the tactical instructions output by the macro-tactical strategy mapping function at time t. and Let represent the local state of the i-th individual's action strategy and its atomic action, respectively. This represents the comprehensive reward signal returned by the virtual simulation environment (also known as the reward or reward signal).

[0085] In practical implementation, the parameters of the macro-tactical strategy mapping function The update aims to optimize long-term tactical effectiveness and style consistency. By evaluating the cumulative performance of tactical instruction sequences across multiple time steps, the macro-tactical strategy mapping function gradually learns to generate tactical instructions that conform to the target team's tactical style under different macro-game conditions. This update process can be formally represented as optimizing the following objective function in the desired sense. ,in As a discount factor, This represents the macro-tactical reward function (also known as the high-level reward function) used to evaluate the effectiveness of tactical instructions at the overall tactical level. The macro-tactical reward function measures the degree to which the generated tactical instructions match the target team's tactical style constraints in terms of space utilization, pace of advancement, defensive positioning, and offensive-defensive transitions.

[0086] Meanwhile, the parameters of individual action strategies The update is performed using reinforcement learning, with the optimization objective of maximizing the cumulative reward related to the game task while satisfying tactical command constraints. The optimization objective function for updating the parameters of the individual action policy can be expressed as follows: ,in, This indicates the degree to which the game objectives have been achieved, as reflected by the virtual football simulation environment according to the game rules (also known as the basic reward signal related to the game objectives). This represents an evaluation function used to measure the consistency between an individual's combined action-strategy behavior and tactical instructions. These are weighting parameters used to balance the constraints of competition objectives and tactical consistency. Through this method, individual action strategies, while collaboratively achieving competition objectives, allow the overall behavior to statistically approximate the macroscopic tactical intent described by the macroscopic tactical strategy mapping function.

[0087] In step 6, steps 4 and 5 are executed iteratively to optimize the parameters of the macro-tactical strategy mapping function and the parameters of the individual action strategy until the preset training termination condition is met, and the parameters of the iteratively updated macro-tactical strategy mapping function and the parameters of the individual action strategy are output.

[0088] In this embodiment, the preset training termination condition includes at least one of the following conditions: the number of training rounds or interaction steps reaches a preset upper limit; the performance evaluation index in the virtual competition changes less than a preset threshold over multiple consecutive training cycles; the macro-tactical reward function converges or tends to stabilize; and the consistency evaluation function converges or tends to stabilize.

[0089] Specifically, in each round of the training iteration process, the macro-tactical strategy mapping function generates a sequence of tactical instructions based on the current macro-match state. Multiple individual action strategies interact with the virtual simulation environment under the constraints of the tactical instructions, forming an interactive trajectory consisting of the macro-level game state, tactical instructions, the local state of individual action strategies, individual actions, and rewards. Based on the aforementioned interaction trajectory, the parameters of the macro-tactical strategy mapping function are... Parameters of individual action strategies The update process involves updating the parameters of the macro-tactical strategy mapping function with the optimization objective of long-term tactical effectiveness and overall match performance, while updating the parameters of individual action strategies with the objective of maximizing cumulative reward while satisfying tactical command constraints. This parameter update process can be represented as iteratively optimizing the strategy parameters under the expectation of cumulative reward.

[0090] The training process terminates when preset training termination conditions are met. These preset termination conditions include, but are not limited to: the number of training rounds or interaction steps reaching a preset upper limit; the performance evaluation indicators in the virtual game changing by less than a preset threshold over several consecutive training cycles (also known as multiple consecutive training cycles); the macro-tactical reward function converging or stabilizing; the consistency evaluation function converging or stabilizing; or the deviation between the distribution of the virtual team's key tactical indicators and the target team's style constraint set being less than a preset threshold. By setting the above termination conditions, it is ensured that the macro-tactical strategy mapping function and the individual action strategy model obtained after training achieve the expected results in both game performance and tactical style consistency, thereby obtaining the final strategy model used to drive the virtual team's decision-making. The parameters of the iteratively updated macro-tactical strategy mapping function and the parameters of the individual action strategy are output.

[0091] Figure 3 This is a block diagram of an electronic device 300 according to an embodiment of the present disclosure.

[0092] Reference Figure 3 An electronic device 300 according to embodiments of the present disclosure may include a processor 310 and a memory 320. The processor 310 may include (but is not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 320 may store computer programs to be executed by the processor 310. The memory 320 includes high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 310 executes the computer program stored in the memory 320, the layered adversarial deduction method described above can be implemented.

[0093] Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store computer programs and any associated data, data files, and data structures in a non-transitory manner and to provide the computer programs and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer programs. In one example, the computer programs and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer programs and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0094] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A method for deducing hierarchical adversarial scenarios, wherein the method is used to train a virtual AI team with the style of a target team, characterized in that, The deduction method includes: Step 1: Obtain the real football match data of the target team, which includes text-based event data and tracking data; Step 2: Based on the real football match data, calculate the tactical indicators of the target team under different match phases, and statistically aggregate the tactical indicators from multiple historical matches to obtain a style constraint set. The match phase refers to the tactical stage divided according to the status of possession and tactical objectives. The tactical indicators are used to quantitatively describe the tactical behavior characteristics of the team. Step 3: Establish a macro-tactical strategy mapping function and initialize the parameters of the macro-tactical strategy mapping function. The macro-tactical strategy mapping function is used to generate tactical instructions to describe the team's overall tactical intentions in the current game phase based on the game state and style constraint set. The game state includes motion state information that characterizes the motion characteristics of the participants in the game and context information that characterizes the game process. Step 4: In the virtual football simulation environment, extract the macro-level match state from the match state of the virtual football simulation environment. Based on the macro-level match state and the style constraint set, generate the tactical instructions through the macro-level tactical strategy mapping function. The virtual football simulation environment includes multiple execution entities that can independently make action decisions as participants in the virtual match, and the interactions between these execution entities. The macro-level match state is obtained by aggregating the position, speed, and ball possession information of the execution entities and is used to characterize the team's overall formation structure, ball possession status, and spatial distribution. Step 5: In the virtual football simulation environment, each execution entity independently selects and executes an individual action through an individual action strategy. The individual action strategy is a strategy function by which the execution entity selects the individual action to be executed based on the tactical instructions and local state. The execution entity interacts with the virtual football simulation environment by executing the individual action to obtain trajectory data. Based on the trajectory data, the parameters of the macro-tactical strategy mapping function and the parameters of the individual action strategy are updated using reinforcement learning methods. The local state is a subset of the game state, including the position, speed, and ball possession information of the execution entity and other execution entities within a predetermined range of the execution entity. The parameter update of the macro-tactical strategy mapping function is optimized based on the degree of matching between the generated tactical instructions and the style constraint set. The parameter update of the individual action strategy is optimized based on the consistency between the game task objective and the tactical instructions. Step 6: Iteratively execute steps 4 and 5 to optimize the parameters of the macro-tactical strategy mapping function and the parameters of the individual action strategy until the preset training termination condition is met, and output the iteratively updated parameters of the macro-tactical strategy mapping function and the parameters of the individual action strategy.

2. The method according to claim 1, characterized in that, The steps for initializing the parameters of the macro-tactical strategy mapping function include: initializing the parameters of the macro-tactical strategy mapping function based on the style constraint set; or using tactical decision samples from real match data to perform supervised pre-training on the parameters of the macro-tactical strategy mapping function to initialize the parameters of the macro-tactical strategy mapping function; or using a random initialization method to initialize the parameters of the macro-tactical strategy mapping function.

3. The method according to claim 1, characterized in that, The steps for extracting the macro-level match state from the match state of the virtual football simulation environment include: extracting macro-level features reflecting the team's overall formation structure, ball possession status, and space occupation distribution from the environmental state containing the position, speed, and ball possession information of the executing entity.

4. The method according to claim 1, characterized in that, The tactical instructions are encoded into a set of structured macro-control parameters to describe the team's overall space utilization, offensive pace, defensive positioning, and direction of offensive and defensive transitions in the current game phase.

5. The method according to claim 4, characterized in that, The macro-control parameters are parameters for regulating the geometric topology of the team's overall spatial structure. The macro-control parameters include: a center of gravity position parameter for defining the team's overall center of gravity coordinates; a stretching-compression parameter for defining the dispersion or compactness of the team's player distribution; and a parameter for defining the team's distribution weight in the lateral space or the width utilization preference of the target area.

6. The method according to claim 1, characterized in that, The objective function for optimizing the parameter update of the macro-tactical strategy mapping function is: ,in, The parameters of the macro-tactical strategy mapping function are... Expressing expectations, As a discount factor, This is a macro-tactical reward function used to measure the degree to which the generated tactical instructions match the set of style constraints in at least one of the following aspects: space utilization, advance pace, defensive positioning, and offensive-defensive transitions. This indicates the macroscopic state of the competition. This refers to the tactical instruction given at time t.

7. The method according to claim 6, characterized in that, The objective function for optimizing the parameter update of the individual action strategy is: ,in, For individual action strategy parameters, The degree to which the game objectives have been achieved, as fed back by the virtual football simulation environment according to the game rules. This is a consistency evaluation function used to measure the consistency between the individual's actions and the tactical instructions. For weight parameters, This represents the individual action of the i-th executing entity at time t.

8. The method according to claim 7, characterized in that, The preset training termination conditions include at least one of the following: the number of training rounds or interaction steps reaches a preset upper limit; the performance evaluation index in the virtual competition changes less than a preset threshold over multiple consecutive training cycles; the macro-tactical reward function converges or tends to stabilize; and the consistency evaluation function converges or tends to stabilize.

9. The method according to claim 1, characterized in that, The event data includes one or more of passing, shooting, and interception, and the tracking data includes the spatial coordinates, velocity vector, ball position, and possession status of each player at each moment.

10. The method according to claim 1, characterized in that, The style constraint set can be represented as a probability distribution, a weight vector, a threshold rule, or any combination thereof.

11. The method according to claim 1, characterized in that, The game phase includes one or more of offensive phase, defensive phase, and transition phase, wherein the transition phase includes offensive-to-defensive phase and defensive-to-offensive phase.

12. The method according to claim 11, characterized in that, The tactical metrics include one or more of the following in the attacking phase: gradual action rate, half-space occupation rate, and packing rate; and / or one or more of the following of the following in the defensive phase: pressing intensity and defensive line height; and / or one or more of the following of the following in the transition phase: rebuilding time, counter-pressing success rate, average speed of team retreat, and cumulative expected goals of the opponent.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program; wherein, when the processor executes the computer program, it implements the deduction method as described in any one of claims 1-12.

Images