An electromagnetic environment simulation method based on armored vehicle cluster communication

Abstract

The application discloses an electromagnetic environment simulation method based on armored vehicle cluster communication, which comprises the following steps: constructing a structured behavior library containing various attack behaviors and defense strategies, and defining a cross-layer state vector running through the physical, network and task three layers; in the simulation process, attack and defense intelligent agents driven by multi-agent reinforcement learning make game decisions according to a payoff function associated with the success or failure of a task, and select actions in the behavior library; attack and defense events are executed through a discrete event simulation engine, and an electromagnetic environment containing the influence of an antenna posture and cross-layer state conduction are accurately evolved; after the simulation ends, a causal inference algorithm is applied to analyze massive time series data, a causal relationship diagram is constructed, and automatic root diagnosis of task failure is realized. The method aims to solve the problems that traditional simulation cannot simulate intelligent game, is hierarchically fragmented and lacks deep diagnosis capability, and can realize high-fidelity intelligent game simulation and causal mechanism insight.

Description

Technical Field

[0001] This invention relates to the field of wireless communication network simulation technology, and in particular to an electromagnetic environment simulation method based on armored vehicle cluster communication. Background Technology

[0002] With the deepening development of modern military informatization, ad-hoc networks with armored vehicles as nodes have become the infrastructure for achieving battlefield situational awareness sharing, command coordination, and joint operations. To ensure the reliability and survivability of such networks in real combat environments, high-fidelity simulation testing during the research and development and deployment phases is crucial. Current network simulation technologies, such as those based on platforms like NS-3 and OPNET, can model and evaluate the physical layer channel characteristics, MAC layer protocol efficiency, and network layer routing performance of mobile ad-hoc networks. Some advanced simulations even integrate 3D Geographic Information Systems (GIS) to simulate the impact of terrain obstruction on signal propagation and, to a certain extent, simulate simple, static electronic jamming scenarios, providing a powerful tool for verifying the basic performance of communication systems.

[0003] However, existing simulation technologies are increasingly showing limitations when dealing with future intelligent and collaborative battlefield environments. First, traditional simulation methods mostly employ scripted attack models, failing to simulate the intelligent counter-attack behavior of attackers dynamically adjusting to the defender's countermeasures. This results in a severe disconnect between simulation results and the complex game processes of real attack and defense scenarios. Second, simulation evaluation dimensions are relatively singular, typically limited to network service quality (QoS) indicators such as throughput and latency. This makes it difficult to directly correlate micro-level changes in network performance with the macro-level operational effectiveness of cluster "mission completion," meaning it cannot effectively answer core questions like "Can the network support mission success?" Third, existing simulation methods suffer from hierarchical fragmentation in their models. Attack methods such as physical layer interference, network layer routing spoofing, and application layer data pollution are often analyzed in isolation, failing to reveal the "combined kill chain" effect generated by the coordinated action of attacks at different levels, and also failing to assess the comprehensive effectiveness of cross-layer defense strategies. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides an electromagnetic environment simulation method based on armored vehicle cluster communication to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an electromagnetic environment simulation method based on armored vehicle cluster communication, comprising:

[0007] Construct a structured behavior library containing various attack behaviors and defense strategies. The behaviors and strategies are configured to modify the cross-layer state vector of an armored vehicle node. The cross-layer state vector includes state parameters characterizing the physical layer, network layer, and mission layer of the armored vehicle.

[0008] During the simulation, agents representing the attacker and defender select the attack behavior and defense strategy for the current round from the structured behavior library based on their respective observed state information and a preset payoff function associated with the success or failure of the task.

[0009] The attack behaviors and defense strategies are transformed into simulation events, and the simulation engine executes the simulation events to update the cross-layer state vector of the armored vehicle node and evolve the electromagnetic environment during the simulation process.

[0010] After the simulation ends, based on the cross-layer state vector time series data recorded during the simulation and the selected attack behavior and defense strategy, a causal inference algorithm is applied to construct a directed acyclic graph representing the causal relationship between the state parameters, and the root cause simulation event that caused the preset task to fail is identified based on the directed acyclic graph.

[0011] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, wherein: in constructing the structured behavior library, each attack behavior or defense strategy is defined as a data structure, which includes:

[0012] An identifier that specifies the protocol layer in which the attack behavior or defense strategy takes effect;

[0013] A set of preconditions for the execution of the aforementioned attack behavior or defense strategy;

[0014] A set of preset rules for modifying specific state parameters in the cross-layer state vector.

[0015] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the payoff function value is determined by a weighted combination of a task success rate indicator, a simulation resource consumption quantification value, and a communication behavior detection probability quantification value; wherein the task success rate indicator has a positive weight, and the other two have negative weights.

[0016] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the probability quantification value of the communication behavior being detected by the enemy is obtained by calculating the information divergence between the current communication behavior feature distribution and the historical communication behavior feature distribution.

[0017] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the process of the agent selecting attack behavior and defense strategy is implemented through a multi-agent reinforcement learning algorithm; wherein, each agent outputs a deterministic action for the current round based on its own partially observable state information using a pre-trained policy network.

[0018] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the electromagnetic environment during the evolution simulation process includes:

[0019] When the simulated event is physical layer interference, according to the power spectral density model of the interference, an interference power term is added in addition to the original noise term during the calculation of the signal-to-interference-plus-noise ratio of the nodes in the affected area.

[0020] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the calculation process of the signal-to-interference-plus-noise ratio further includes:

[0021] Based on the real-time attitude of the armored vehicle node, the corresponding antenna gain value is queried and applied from a database that pre-stores the three-dimensional radiation patterns of the vehicle's antennas under different attitudes.

[0022] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the process by which the simulation engine executes the simulation event includes:

[0023] When the state parameter of a certain protocol layer changes due to the execution of a simulation event, and the change satisfies the triggering condition of a simulation event of another protocol layer, a new simulation event of the other protocol layer is automatically generated and scheduled.

[0024] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the causal inference algorithm employs a constraint-based causal discovery algorithm. The algorithm determines the correlation between state parameters by performing conditional independence tests on the time series data and applies orientation rules to determine the causal direction of the correlation, thereby forming the directed acyclic graph.

[0025] As a preferred embodiment of the electromagnetic environment simulation method based on armored vehicle cluster communication described in this invention, the process of identifying the root cause simulation events leading to the failure of the preset task includes:

[0026] In the directed acyclic graph, starting from the node representing task failure, a reverse tracing is performed along a path with causal relationships until one or more root nodes without upstream input nodes are reached, generating a diagnostic report containing the root node, the tracing path, and the task failure result.

[0027] Compared with existing technologies, the beneficial effects of this solution are:

[0028] 1. This invention introduces a multi-agent reinforcement learning (MARL) framework to construct an offensive-defensive game model based on a partially observable Markov decision process. In this model, the agent's decision-making is guided by a comprehensive payoff function directly related to task success or failure, resource consumption, and stealth. This breaks away from the fixed, predictable script-based attacks of traditional simulations, enabling realistic simulation of highly dynamic and adaptive adversarial behaviors. This transforms simulation from network performance evaluation into a more realistic tactical-level game theory exercise.

[0029] 2. By defining cross-layer state vectors that span the physical layer, network layer, and task layer, and incorporating a cross-layer event triggering and propagation mechanism, the problem of fragmented protocol stack layers in traditional simulations is solved. This enables a low-level physical phenomenon (such as increased interference or antenna attitude change) to automatically and in a chain-like manner propagate upwards and trigger a chain reaction in higher-level protocols and task states, thereby accurately reproducing and evaluating the "combined kill chain" effect of cross-layer cooperative attacks and the effectiveness of comprehensive defense strategies.

[0030] 3. Furthermore, this invention introduces a causal inference algorithm after the simulation, transforming the simulation time-series data into a directed acyclic graph representing the causal relationships between system variables. Based on this graph, it is possible to automatically and interpretably trace back the root cause of system failure (such as a specific attack) starting from the final result of "task failure," providing in-depth analysis and decision support capabilities for system design optimization, vulnerability analysis, and tactical research. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0032] Figure 1 This is a flowchart illustrating the overall process of an electromagnetic environment simulation method based on armored vehicle cluster communication according to an embodiment of the present invention. Detailed Implementation

[0033] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0034] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0035] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0036] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0037] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0038] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0039] Example 1

[0040] Reference Figure 1 This is the first embodiment of the present invention, which provides an electromagnetic environment simulation method based on armored vehicle cluster communication, including:

[0041] S1. Construct a structured behavior library containing various attack behaviors and defense strategies. The behaviors and strategies are configured to modify the cross-layer state vector of the armored vehicle node. The cross-layer state vector includes state parameters representing the physical layer, network layer, and mission layer of the armored vehicle.

[0042] It should be noted that building a high-quality behavior library is a prerequisite for realizing game simulation. Therefore, the purpose of this step is to formalize and structure the complex, cross-protocol layer electromagnetic network attack and defense behaviors, so that they can be invoked by subsequent agents and parsed by the simulation engine.

[0043] Furthermore, the process of building this structured behavior library is usually divided into two sub-steps: defining cross-layer state vectors and defining the data structures for attack behaviors and defense strategies.

[0044] Furthermore, to comprehensively characterize the operational state of armored vehicle nodes in complex electromagnetic environments, this invention defines a multi-dimensional cross-layer state vector. This cross-layer state vector integrates key parameters from physical devices to upper-level tasks and is the target of all offensive and defensive actions. This cross-layer state vector consists of physical layer state vectors, network layer and data link layer state vectors, and task layer state vectors.

[0045] Specifically, the physical layer state vector includes, but is not limited to: the three-dimensional position coordinates, velocity, attitude (pitch, roll, yaw), antenna pointing, current transmit power, operating frequency, channel occupancy status, and received signal-to-interference-plus-noise ratio (SINR) of the armored vehicle node.

[0046] It should be noted that the physical layer state vector directly determines the establishment and quality of the communication link.

[0047] Specifically, the network layer and data link layer state vectors include, but are not limited to: the routing table contents of the armored vehicle node, the list of neighboring nodes, the MAC layer protocol state (such as the backoff window size of CSMA / CA), packet forwarding table entries, the number of packets to be sent / received in the queue, the average end-to-end latency of packets, and the packet loss rate.

[0048] It should be noted that the state vectors of the network layer and data link layer reflect the connectivity of the network topology and the efficiency of data transmission.

[0049] Specifically, the mission layer state vector includes, but is not limited to: the mission type currently being performed by the armored vehicle node (such as reconnaissance, fire guidance, and coordinated maneuver), the integrity verification value of key mission data, the received sequence of coordinated instructions, and the evaluation value of the current armored vehicle node's contribution to the mission of the entire armored vehicle cluster.

[0050] It should be noted that this mission-level state vector is designed to directly link network performance with ultimate operational effectiveness.

[0051] Furthermore, for each attack behavior or defense strategy in the behavior library, it is defined as a standardized data structure to improve the scalability and maintainability of the behavior library. A typical attack behavior or defense strategy data structure usually includes an effective protocol layer identifier, preconditions, and a set of modification rules.

[0052] Specifically, in this embodiment, the effective protocol layer identifier is an enumeration value or string, namely PHY (physical layer), NET (network layer), or TASK (task layer). This is intended to explicitly indicate which sub-part of the cross-layer state vector the attack or defense strategy applies to.

[0053] Specifically, in this embodiment, the preconditions are a set of logical expressions. It is important to emphasize that the simulation engine only allows the attack or defense strategy to be executed if it verifies that the cross-layer state vector of the current armored vehicle node satisfies all the preconditions. For example, the precondition for a "black hole attack" might be that "the node is the intersection of two critical paths," meaning its betweenness centrality in the network topology is greater than a certain threshold.

[0054] Specifically, in this embodiment, the modification rule set is a set of functions or instructions that specify how to modify the cross-layer state vector of the target armored vehicle node. These rules can be deterministic or random.

[0055] For example, this embodiment provides three examples of the data structures for the aforementioned attack behaviors or defense strategies, as follows:

[0056] Example 1: Physical layer blocking interference attack behavior: Its effective protocol layer is identified as physical layer. The prerequisite is that the attacker's interference power is greater than ours, that is, the attacker has sufficient interference power. At this time, the modified rule set is applied to all enemy armored vehicle nodes in the affected area, and the signal-to-interference-plus-noise ratio (SINR) in their physical layer state will be updated.

[0057] Example 2, Network Layer AODV Route Spoofing Attack: This attack is identified as occurring at the network layer. The prerequisite is receiving a Route Request (RREQ) packet. The modified rule set immediately forges a Route Response (RREP), claiming to have the shortest path to the destination (e.g., setting the hop count to 1). Subsequently, the routing table entry recorded in this fake RREP is added to the routing table of the attacker's network layer state (i.e., the network layer and data link layer state vectors), and the RREP is broadcast to deceive neighboring nodes into incorrectly forwarding packets to the attacker.

[0058] Example 3: Cross-layer collaborative spectrum hopping defense strategy: Its effective protocol layer is identified between the physical layer and the network layer. The prerequisite is that the average packet loss rate exceeds a preset average packet loss rate threshold, and power interference is detected. In this case, the rule set modification performs two operations: First, a physical layer modification, where the operating frequency parameter in the physical layer state vector is switched to a new frequency with minimal interference selected from the prepared operating frequency table. Second, a network layer modification, where a network control signaling broadcast event is triggered in the network layer and data link layer state vectors to notify all neighboring nodes to perform a synchronized frequency switch.

[0059] It should be noted that by following the above steps, a comprehensive and parameterized library of offensive and defensive behaviors can be constructed. This library is not only a set of parameter definitions, but also provides a standardized "toolkit" for both offensive and defensive agents to select appropriate parameters based on the current battlefield situation.

[0060] S2. During the simulation, agents representing the attacker and defender select the attack behavior and defense strategy for the current round from a structured behavior library based on their respective observed state information and a preset payoff function that is associated with the success or failure of the task.

[0061] It should be noted that this step aims to model the simulation process as a dynamic game under incomplete information. By introducing a reinforcement learning-based agent, the present invention can simulate the learning, adaptation, and decision-making of both attackers and defenders in a constantly changing environment, thereby breaking away from the fixed and predictable scripted behavior in traditional simulations and improving the realism and adversarial nature of the simulation.

[0062] Furthermore, the entire simulation process is abstracted into a multi-agent game model. For each decision round (i.e., a discrete time step in the simulation process), both the attacker and defender need to make a decision. Since in a real battlefield environment, neither side can obtain global and complete state information (for example, the defender cannot directly observe the attacker's intentions and resources, and the attacker cannot accurately know the defender's complete network state), this embodiment uses a partially observable Markov decision process (POMDP) ​​to model this problem.

[0063] Specifically, the POMDP of an agent (which can be either an attacker or a defender) can be represented by an experience tuple. The elements of this tuple consist of the global state space, the agent's action space, the state transition function, the agent's reward function, the agent's observation space, and the observation function. The global state space is composed of the cross-layer state vectors of all armored vehicle nodes. The agent's action space is the set of available attack behaviors / defense strategies specified for it from the structured behavior library constructed in step S1. The state transition function describes the transition of the system state from one action to the next under joint actions (i.e., the combination of actions of all agents). (At the start of a certain decision round t, the global state of the entire armored vehicle cluster) transitions to... (The probability of the new global state that the system presents at the start of the next decision round t+1 after the attacker and defender agents choose actions (an attack behavior and a defense strategy) in decision round t, and the effects of these actions are calculated and applied by the simulation engine. This transition process is specifically executed by the subsequent simulation engine. The agent's reward function is its output value (i.e., the payoff function value) in each round, used to guide the agent's learning and decision-making. The agent's observation space is a partial mapping of the global state. The observation function is defined as... The probability that the agent obtains the observation.

[0064] It is important to note that in the state transition function, even if the action is deterministic, the transition from... Transferred to There will still be a probability involved because simulation environments often have randomness. For example, even with fixed transmission power and distance, due to the fading and randomness of wireless channel noise, a single data packet transmission is not 100% successful; it has only one probability of success. The status parameter "whether the data packet was successfully delivered" is probabilistic.

[0065] Furthermore, to ensure that the agent's decision-making objectives align with the overall task performance of the cluster, this invention designs a comprehensive payoff function (i.e., the agent's reward function). This function considers not only immediate gains but also a quantitative assessment of future risks to guide the agent in making more forward-looking decisions. At the end of each decision round t, the payoff function is calculated as follows:

[0066]

[0067] in, The payoff function value is calculated at the end of decision round t. It is a mission success rate metric, which is a normalized value (range [0,1]) used to measure the progress of mission completion in the current state. For example, in an area reconnaissance mission, this metric can be "the proportion of the area of ​​the reconnaissance area to the total target area"; in a coordinated strike mission, this metric can be "the number of key targets that have been successfully identified and locked". It is a quantitative value of simulated resource consumption, used to measure the cost of executing the selected attack behavior or defense strategy. For example, for jamming attacks, this quantitative value represents the power consumed; for frequency hopping defense, this quantitative value is the bandwidth and processing overhead occupied by the handover signaling. This is a quantified value representing the probability that communication behavior will be detected by the enemy. It is used to assess the stealth of our communication behavior and is key to achieving low probability of intercept communication simulation. This value is obtained by calculating the information divergence between the distribution of communication behavior characteristics at the current time and at historical time points. In this embodiment, the Kullback-Leibler (KL) divergence can be used for calculation.

[0068]

[0069] in, It is the distribution of communication behavior characteristics (such as frequency usage histogram, transmit power probability density function, etc.) statistically analyzed in the current decision round t. It is the historical average distribution of communication behavior characteristics. This refers to the KL divergence. It's important to explain that KL divergence is a concept in information theory used to measure the difference between two probability distributions. This KL divergence is asymmetric, meaning... In this formula, the distribution of communication behavior characteristics at the current moment is measured. Relative to historical average communication behavior feature distribution The information gain is such that when the KL divergence value increases significantly, it means that the current communication mode deviates greatly from the normal mode, making it easier for the enemy to detect and identify. When the KL divergence is 0, it means that there is no abnormality and the risk of being detected is the lowest. It is the set of all possible values ​​for the communication behavior characteristic. For example, if the communication behavior characteristic is the transmit power level, then... It is the set of all transmit power levels; if the communication behavior is characterized by frequency usage, then this... It is the collection of all frequency bands. This is a specific value in the set of communication behavior characteristics, for example, the 3rd frequency band. These are the weighting coefficients of the three items mentioned above, all of which are positive numbers, and By adjusting these three weights, decision preferences under different operational doctrines can be simulated (e.g., emphasizing completing the mission at all costs, or emphasizing the preservation of strength and prioritizing concealment).

[0070] Furthermore, since the core of agent decision-making is the multi-agent reinforcement learning (MARL) algorithm, this embodiment preferably employs an algorithm based on the "Centralized Training with Decentralized Execution" (CTDE) framework.

[0071] Specifically, in the centralized training phase (offline phase): Before the simulation begins, all agents (including attackers and defenders) engage in numerous rounds of self-play in a shared training environment. During training, the battlefield situation is randomly initialized at the start of each round, for example, the initial positions of each node, the mission objective, and the initial fault states of some nodes are randomly set. In each training iteration, all agents (attackers and defenders) generate exploratory actions based on their current policy network and a noise term (e.g., Gaussian noise) to increase exploration. The simulation engine executes these actions and the current system state, calculating the reward for each agent. Complete experience tuples are stored in a global "experience replay pool." Then, the critic network and policy network are updated. For the critic network, a small batch of experience data is randomly sampled from the experience replay pool, and for each sample, the target Q-value for the next state is calculated using the target network. For the policy network, the policy network parameters of each agent are updated using the previously updated critic network and the sampled policy gradients. The training process is repeated until the preset number of training rounds is reached, or the average reward value of all agents converges to a stable level. At this point, the policy network parameters of all agents are saved to form a "pre-trained policy network" for direct use in the online simulation phase (i.e., the distributed execution phase).

[0072] It should be noted that each agent has a policy network. A network of critics The input to the critic network includes not only the agent's own observations and actions, but also the observations and actions of all other agents. This allows the critic network to learn a global joint action-value function, thus solving the learning challenge in non-stationary environments and accurately guiding the policy network's updates. The policy network's update gradient follows these principles:

[0073]

[0074] in, For intelligent agents The parameters of the policy network need to be explained. The policy network is a deep neural network whose function is to receive an observation as input and then output a specific action. This is the set of all weights and biases in the neural network, and our goal is to optimize these through training. This makes neural networks "smarter". The long-term expected return, or the objective function of the policy, is simply the expected value of the total cumulative reward (payoff function value) that the agent can obtain from the current execution of its policy until the end of the simulation. The goal is to maximize this reward. This means teaching the agent how to achieve the highest possible total score. yes right The gradient, as explained, is a vector that points to the state that makes... The fastest growing direction, during the training process, allows Make a small update along the direction of this gradient ( ,in, (This is the learning rate), which can optimize the policy network. This is represented as the expectation. Since state transitions and agent exploration can both be random, we need to calculate the average performance across all possible trajectories. In practice, this expectation is usually approximated by randomly sampling a batch of historical data from the experience replay pool. It is the joint action of all N intelligent agents. It is the output of the critic network of the agent, which is an action-value function. The input of the critic network is the global state. Joint actions of all intelligent agents The output is a Q-value, which evaluates the performance of the system in the global state. After making this joint action, what is the long-term expected return that the agent can obtain from this moment into the future? Since the critic network can see global information, it can accurately evaluate the quality of each agent's action. Q is the gradient of the agent's joint actions, that is, how the Q value (i.e., long-term reward) will change if some of the agent's actions change. It indicates the direction for improving certain actions. For example, if the gradient is positive, it means that increasing the "magnitude" of certain actions (such as increasing the power of interference) will bring higher rewards. It is a substitution operation, designed to substitute the previously calculated gradients. Replace a certain action in the network with the current policy network. The specific action output. It is the output of the agent's policy network, that is, the policy network. Receive local observations from the intelligent agent As input, it directly outputs a deterministic action. It is the gradient of the action output by the policy network with respect to the policy network parameters, which can be automatically calculated through backpropagation of the neural network.

[0075] Specifically, in the distributed execution phase (online simulation phase): in each decision round t, the trained agent only needs to input a portion of its own observable state information into its already fixed policy network to quickly output a currently optimal deterministic action (i.e., select a specific action or policy from the action library):

[0076]

[0077] It should be noted that the above-mentioned "centralized training and decentralized execution" approach eliminates the need for complex value assessments and multi-step simulations online, and the decision-making speed is fast, fully meeting the requirements of real-time or semi-real-time simulation. It simulates the real-world situation where each unit makes independent decisions based on the local battlefield situation.

[0078] S3. Transform attack behaviors and defense strategies into simulation events, and have the simulation engine execute these events to update the cross-layer state vector of the armored vehicle node and evolve the electromagnetic environment during the simulation process.

[0079] It should be noted that this step is mainly responsible for parsing the abstract offensive and defensive "actions" selected by the agent in step S2 into specific, time-sequenced "event sequences" that the simulation engine can understand and execute, and calculating the impact of these events on the state of each object in the simulation world.

[0080] Furthermore, in step S2, since the actions output by the agent (such as "execute the jamming attack as A05" or "execute the frequency hopping strategy as D02") are high-level instructions, these instructions need to be translated into low-level, executable simulation events. A simulation event is typically a data structure containing (execution time, event type, event parameters, and target object).

[0081] Furthermore, this data structure includes event transformation and event scheduling.

[0082] Specifically, for event transformation, the transformation process is based on the behavior library data structure defined in step S1. For example, when the agent selects the behavior "physical layer blocking interference attack", the transformer will read the "modification rule set" of that behavior and generate one or more simulated events. For example, it generates an interference initiation event, which includes the interference source ID, interference power spectral density model, center frequency, bandwidth, and range of action.

[0083] Specifically, for event scheduling, the transformed events are sent to the event queue of the simulation engine. It's important to explain that the simulation engine is a typical Discrete Event Simulation (DES) engine, maintaining a time-ordered event queue. The simulation engine always retrieves the event with the smallest timestamp from the head of the queue for execution. During execution, new events may be generated, and these new events are inserted into the queue at a future point in time. This cycle repeats, advancing the simulation time and ensuring that simulation events are executed strictly according to causal sequence.

[0084] Furthermore, when the simulation engine retrieves and executes an event from the event queue, it calls the corresponding model to calculate the impact of the event on the system state and updates the cross-layer state vector of the affected node.

[0085] Specifically, for events at the network layer or task layer, the execution process is relatively straightforward. For example, when executing an "AODV route spoofing attack" event, the simulation engine will directly modify the routing table entry in the target node's network layer state vector, pointing it to the attacker.

[0086] Specifically, the execution of physical layer events is the most complex because it directly relates to the dynamic changes in the electromagnetic environment. When the simulated event being executed is a physical layer disturbance, its impact on the evolution of the electromagnetic environment is as follows:

[0087] The simulation engine first calculates the power of the interference signal at each location in the simulation space based on the parameters of the interference event (such as the location, power, and antenna pattern of the interference source) and a preset electromagnetic wave propagation model (such as the free space loss model, Rayleigh fading model, or a more complex ray tracing model). Then, for each receiving node within the affected area, the simulation engine calculates its signal-to-interference-plus-noise ratio (SNR). It will be dynamically updated at that time. The original signal-to-interference-plus-noise ratio (SIR) calculation formula is:

[0088]

[0089] in, It is a node The power of the received useful signal.

[0090] After the interference event is executed, the formula is modified to:

[0091]

[0092] Among them, the increased This refers to the external interference power term introduced by the interference event, used to simulate the degrading effect of suppressive interference on the communication link.

[0093] Furthermore, to further enhance the physical realism of the simulation, the useful signal power in the above signal-to-interference-plus-noise ratio calculation process is... It's not just simple attenuation based on distance, but rather the signal loss from the transmitting node. to receiving node The received power is calculated using the following formula:

[0094]

[0095] in, That is the transmission power. It's path loss, it's distance. The function. and These are the antenna gain values ​​for the transmitting and receiving antennas, respectively. It is important to emphasize that these gain values ​​are not fixed constants, but rather functions of the angle. and These represent the azimuth and elevation angles of the signal's emission / incidence direction, respectively.

[0096] Furthermore, armored vehicles experience severe jolting and constant attitude changes when traversing rough terrain. For directional antennas, even minute attitude changes can cause the main lobe to deviate from the target object, resulting in a sudden drop in antenna gain of tens of dB and a momentary communication interruption. Traditional simulations ignore the influence of attitude, significantly overestimating link stability. Therefore, this invention, by introducing attitude-pattern linkage into the simulation engine, can reproduce this "attitude blinding" effect, thereby imposing more stringent requirements on the dynamic adaptability and robustness of the communication system.

[0097] Specifically, at each time step, the simulation engine calculates the signal transmission and reception angles based on the real-time attitude (pitch, roll, yaw) and relative position of the armored vehicle node in the physical layer state vector. Then, it queries and applies the corresponding antenna gain value from a database that stores the three-dimensional radiation patterns of the vehicle's antennas under different attitudes.

[0098] Furthermore, the simulation engine also incorporates a cross-layer event triggering and propagation mechanism.

[0099] Specifically, after the simulation engine executes an event and updates a certain state parameter of a node, it will immediately check whether the state change meets the trigger conditions preset in the behavior library or other models.

[0100] For example, the transmission from the physical layer to the network layer: When a physical layer blocking interference event is executed, causing the signal-to-interference-plus-noise ratio (SINR) state parameter value of node A to fall below a certain threshold (e.g., -5dB), the simulation engine detects the change in SINR and checks whether this change triggers a new event. If the simulation engine finds a rule in the data link layer: "When SINR is below -5dB, the link layer determines that the link is disconnected," then the simulation engine will update the neighbor list in node A's network layer state vector and remove the corresponding neighbor nodes. Since the change in the neighbor list is a network layer event, it will also automatically trigger a routing protocol (such as AODV) to begin route repair, generating a new route request event and adding it to the event queue.

[0101] It should be noted that through the cross-layer event triggering and propagation mechanism, different protocol layers can be tightly coupled together, enabling a low-level physical phenomenon (such as interference) to automatically and in a chain-like manner propagate upwards, ultimately affecting network topology and data transmission, and even the state of the task layer. This solves the problem of layer fragmentation in traditional simulations and the inability to effectively simulate "combined kill chains," allowing simulations to reveal the deep-seated, cross-protocol stack vulnerabilities of the system under complex attacks.

[0102] S4. After the simulation ends, based on the cross-layer state vector time series data recorded during the simulation and the selected attack behaviors and defense strategies, a causal inference algorithm is applied to construct a directed acyclic graph representing the causal relationship between state parameters, and the root cause simulation event that caused the preset task to fail is identified based on the directed acyclic graph.

[0103] It should be noted that traditional simulations typically stop at providing large amounts of time-series data and statistical results, but this data alone cannot automatically reveal complex failure chains, especially when multiple offensive and defensive behaviors are intertwined and cross-layer effects emerge. To address this issue, this step aims to transform simulation data into an intuitive causal relationship network by introducing causal inference techniques, thereby enabling automated and interpretable root cause diagnosis of task failures.

[0104] Furthermore, since the raw data recorded in real time by the simulation is high-dimensional and high-frequency, it needs to be processed to be compatible with causal inference algorithms.

[0105] Specifically, the processing procedure is as follows:

[0106] First, the cross-layer state vectors of all armored vehicle nodes at each decision round t during the simulation, along with the actions (attack behaviors / defense strategies) chosen by both attacking and defending agents, are collected. Then, this data is integrated into a large time-series data matrix. Each row of this matrix represents a discrete time step (or decision round), and each column represents a system variable. These system variables include not only continuous values ​​in the state vectors (such as signal-to-interference-plus-noise ratio, position coordinates) but also discrete values ​​(such as message authentication code status, selected attack method). Based on this, the dimension of the matrix can be represented as... ,in, It is the total number of rounds in the simulation. This represents the total number of system variables. Finally, for continuous variables that require conditional independence tests (such as signal-to-interference-plus-noise ratio), they can be discretized based on their physical meaning or through data-driven methods (such as equal-width or equal-frequency binning) to apply statistical tests based on counts and probabilities.

[0107] Furthermore, the present invention employs a constraint-based causal discovery algorithm (such as PC algorithm, FCI algorithm, etc.) to learn the causal structure between variables. The core idea of ​​such algorithms is to "prune" impossible causal relationships through a series of statistical tests. In this embodiment, the classic PC algorithm is used for processing.

[0108] Specifically, the process of using the PC algorithm is as follows:

[0109] (i) At the beginning of the algorithm, assume that there may be causal relationships between all variables and construct a fully connected undirected graph in which every node (variable) is connected to every other node.

[0110] (ii) The algorithm iteratively removes edges that do not satisfy causal relationships. Specifically, for any two variables... and The algorithm checks whether they are in a given (potentially empty) set of variables. They are independent under certain conditions. This conditional independence relation is denoted as... This step is achieved through a conditional independence test. For discrete data, a commonly used test is the G-squared test (G test):

[0111]

[0112] in, They are variables Different values ​​of . It was observed in the simulation data. The number of times they occur simultaneously (frequency). These are the corresponding marginal frequencies.

[0113] It should be noted that the calculated The value follows a chi-square distribution with a specific number of degrees of freedom. By comparing it with the chi-square value corresponding to a preset significance level (e.g., 0.05), it can be determined whether to accept the result. and In the given The original assumption that "time is independent" is true.

[0114] (iii) The algorithm starts from an empty condition set Start (i.e., check) ), gradually increase the condition set The size of the condition set. If in a condition set of arbitrary size... They couldn't refuse. and If there is a connection between them, then the edges are preserved. Otherwise, once one is found Make Then remove the edge. After this process is completed, an undirected graph is obtained, which is called the skeleton of the causal graph.

[0115] (iv) The algorithm determines the orientation of edges in the skeleton by applying a series of orientation rules. Specifically, the algorithm searches for edges of the form... and and There is no edge between them. If we remove the edges in the previous step... The condition set on which it is based does not contain Then this structure can be oriented as Transforming the structure into a "collision structure" is a crucial factor in determining causal directions. Then, after determining all collision structures, other logical rules (such as the Meek rule) are applied to avoid creating new collision structures or directed cycles, thereby determining the directions of more undirected edges. For example, if it is known... and and If there is no direct connection, then it must be directed to To avoid collisions.

[0116] It should be noted that, after the above steps, a Directed Acyclic Graph (DAG) can be generated, denoted as . , where the node set It is all the state parameters and attack / defense behavior variables in the system, edge set This represents the direct causal relationship inferred between all state parameters and attack / defense behavior variables.

[0117] It should be explained that the constraint-based causal discovery algorithm is used because it does not rely on any prior physical models or expert knowledge to predetermine the causal structure, but rather lets the simulation data "speak for itself." This enables the present invention to discover cross-protocol layer "combined kill chains" effects that were not anticipated during the design phase.

[0118] Furthermore, after obtaining the directed acyclic graph representing the global causal relationships, the present invention will take "task failure" as the starting point to perform automated problem diagnosis.

[0119] Specifically, first, locate the node representing "task failure" in the directed acyclic graph. This is typically a task-level variable defined during the data preparation phase. In this embodiment, "task failure" is defined as a Boolean variable, dynamically monitored during simulation. Its value changes from 0 to 1 when any of the following conditions are met:

[0120] Condition 1: Critical information of the cluster (such as a battlefield situation map) fails to be successfully distributed to more than a preset proportion (such as 50%) of nodes within a specified time.

[0121] Condition 2: The average connection time between the designated core node (such as the armored vehicle under the main command) and more than a preset number (such as 7) of its subordinate nodes is lower than the preset threshold (such as 60%) of the total mission time.

[0122] Condition 3: In the task-level state vector, the parameter representing the "area of ​​the reconnaissance area" is still lower than the preset threshold (e.g., 80%) of the target value at the end of the simulation.

[0123] This decision result is added as a new variable column to the time-series data matrix used for causal inference. Then, starting from the failed node, the graph is traversed in the reverse direction of the edges in the directed acyclic graph (e.g., using depth-first search or breadth-first search). This process traces all direct causes pointing to the failed node, as well as indirect causes pointing to these direct causes, forming one or more causal chains. This tracing process continues until one or more root nodes are reached. It is important to emphasize that these root nodes are those nodes in the directed acyclic graph that have no upstream input nodes (i.e., in-degree is 0).

[0124] It should be explained that these root nodes are the "initial drivers" in the causal chain, and their changes are not caused by other variables within the system. In the simulation scenario of this invention, these root nodes are typically attack events chosen by the attacking agent (such as routine interference), sudden changes in the external environment of the system, or an inappropriate initial strategy setting by the defender.

[0125] Furthermore, the system integrates the traceability results into a clear diagnostic report. This report is presented in a readable text format or a graphical interface, and its content is not limited to simple conclusions, but rather includes a visual and textual explanation of the entire failure process.

[0126] Furthermore, the report consists of diagnostic conclusions, key causal pathways, and optional quantitative support.

[0127] Specifically, the diagnostic conclusion should clearly identify one or more root cause simulation events that led to the task failure with the highest confidence level. For example: "Diagnostic conclusion: The direct root cause of the task failure is the AODV routing protocol spoofing attack event executed by the attacker at simulation time 52.3 seconds."

[0128] Specifically, the critical causal path details the complete path from the root node to the node where the mission failed. For example: A spoofing attack against the AODV routing protocol occurred, which corrupted the routing table on node 5. This corruption caused a critical data packet destined for headquarters to be lost during transmission. The loss of this packet severely compromised the integrity of critical information on the battlefield, ultimately leading to the failure of the entire mission.

[0129] Specifically, in terms of quantitative support, the diagnostic report can also include the correlation strength of each edge on the causal path (e.g., the p-value of the conditional independence test or an information theory index), providing a quantitative basis for the "strength" of the causal relationship, which can be used to simulate post-war analysis to distinguish between primary and secondary contradictions.

[0130] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0135] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An electromagnetic environment simulation method based on armored vehicle cluster communication, characterized in that, include: Construct a structured behavior library containing various attack behaviors and defense strategies. The behaviors and strategies are configured to modify the cross-layer state vector of an armored vehicle node. The cross-layer state vector includes state parameters characterizing the physical layer, network layer, and mission layer of the armored vehicle. During the simulation, agents representing the attacker and defender select the attack behavior and defense strategy for the current round from the structured behavior library based on their respective observed state information and a preset payoff function associated with the success or failure of the task. The attack behaviors and defense strategies are transformed into simulation events, and the simulation engine executes the simulation events to update the cross-layer state vector of the armored vehicle node and evolve the electromagnetic environment during the simulation process. After the simulation ends, based on the cross-layer state vector time series data recorded during the simulation and the selected attack behavior and defense strategy, a causal inference algorithm is applied to construct a directed acyclic graph representing the causal relationship between the state parameters, and the root cause simulation event that caused the preset task to fail is identified based on the directed acyclic graph.

2. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 1, characterized in that, In constructing the structured behavior library, each attack behavior or defense strategy is defined as a data structure, which includes: An identifier that specifies the protocol layer in which the attack behavior or defense strategy takes effect; A set of preconditions for the execution of the aforementioned attack behavior or defense strategy; A set of preset rules for modifying specific state parameters in the cross-layer state vector.

3. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 1, characterized in that, The payment function value is determined by a weighted combination of a task success rate indicator, a simulated resource consumption quantification value, and a communication behavior detection probability quantification value; wherein, the task success rate indicator has a positive weight, and the other two have negative weights.

4. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 3, characterized in that, The probability quantification value of the communication behavior being detected by the enemy is obtained by calculating the information divergence between the current communication behavior feature distribution and the historical communication behavior feature distribution.

5. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 1, characterized in that, The process of the agent selecting attack behavior and defense strategy is implemented through a multi-agent reinforcement learning algorithm; in which each agent outputs a deterministic action for the current round based on its own partially observable state information using a pre-trained policy network.

6. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 1, characterized in that, The electromagnetic environment during the evolution simulation process includes: When the simulated event is physical layer interference, according to the power spectral density model of the interference, an interference power term is added in addition to the original noise term during the calculation of the signal-to-interference-plus-noise ratio of the nodes in the affected area.

7. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 6, characterized in that, The calculation process for the signal-to-interference-plus-noise ratio also includes: Based on the real-time attitude of the armored vehicle node, the corresponding antenna gain value is queried and applied from a database that pre-stores the three-dimensional radiation patterns of the vehicle's antennas under different attitudes.

8. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 1, characterized in that, The process by which the simulation engine executes the simulation event includes: When the state parameter of a certain protocol layer changes due to the execution of a simulation event, and the change satisfies the triggering condition of a simulation event of another protocol layer, a new simulation event of the other protocol layer is automatically generated and scheduled.

9. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 1, characterized in that, The causal inference algorithm employs a constraint-based causal discovery algorithm. The algorithm determines the correlation between state parameters by performing conditional independence tests on the time series data, and applies orientation rules to determine the causal direction of the correlation, thus forming the directed acyclic graph.

10. The electromagnetic environment simulation method based on armored vehicle cluster communication as described in claim 9, characterized in that, The process of identifying the root cause simulation events that lead to the failure of the preset task includes: In the directed acyclic graph, starting from the node representing task failure, a reverse tracing is performed along a path with causal relationships until one or more root nodes without upstream input nodes are reached, generating a diagnostic report containing the root node, the tracing path, and the task failure result.

Images