Asynchronous intelligent control method for real-time simulation system

Abstract

The invention discloses an asynchronous intelligent control method for a real-time simulation system. The asynchronous intelligent control method is based on a simulation environment construction module, an asynchronous feedback acquisition module and an intelligent control decision module. The simulation environment construction module is used for loading nodes, channels and model configuration files and defining simulation operation parameters; the module supports node starting, stopping and parameter adjusting operation, and an executable interface is provided for subsequent intelligent control; the asynchronous feedback acquisition module is used for periodically acquiring node states, link quality, throughput rate and time delay indexes through a probe module in a simulation running process, and writing the node states, the link quality, the throughput rate and the time delay indexes into a state cache in an asynchronous mode; the intelligent control decision module is used for carrying out analysis and strategy reasoning on the asynchronously acquired state by utilizing a reinforcement learning or deep reinforcement learning algorithm; the intelligent agent generates an action decision according to the current state, and outputs a node parameter adjustment instruction, an interface control instruction, a power configuration instruction or a flow scheduling control instruction to the environment construction module; the decision-making process can be continuously carried out in simulation operation, and dynamic self-adaptive optimization is achieved.

Description

Technical Field

[0001] This invention belongs to the field of communication network simulation and intelligent control, and specifically relates to an asynchronous intelligent control method for real-time simulation systems. Background Technology

[0002] Existing network simulation systems can be broadly classified into two categories: one is simulation platforms based on Discrete Event Simulation (DES), such as NS-3 and OPNET; the other is real-time simulation platforms based on virtualization and containerization, such as Mininet and EMANE.

[0003] Discrete event-driven simulation platforms, centered on event queues, advance system states by calculating simulation time. They offer advantages such as high precision and repeatability, and are commonly used for protocol verification and algorithm analysis. However, the runtime of these systems is not synchronized with the simulation time, making real-time interaction impossible.

[0004] In recent years, some studies have attempted to incorporate machine learning methods. For example, the NS3-AI framework uses reinforcement learning models to make intelligent decisions about simulated events, but its control process still relies on a cyclical approach of pausing the simulation, calculating the strategy, and then resuming execution. This approach is insufficient to meet the demands of continuous, adaptive, and real-time control.

[0005] In contrast, real-time simulation platforms use virtualization or containerization to synchronize the simulation process with the system clock, enabling real-time interaction with external applications and offering better system scalability and engineering portability. However, real-time simulation systems typically rely on host performance to ensure synchronization, and when the network size increases or the number of nodes grows, time drift or simulation distortion can easily occur. Furthermore, their interfaces are mostly low-level configuration and data stream-based, lacking a unified intelligent control interface, making it difficult to implement real-time injection and feedback optimization of high-level strategies.

[0006] Currently, some research focuses on the intelligentization of real-time simulation systems. A typical example is the SDN simulation environment based on Mininet, which uses Ryu / OpenFlow to obtain flow tables and traffic statistics, and uses machine learning models to detect traffic characteristics in real time. Once an anomaly is detected, the controller issues flow table rules to isolate or rate-limit attack traffic. However, this method is mostly geared towards specific security / detection use cases and lacks a general policy expression and distribution mechanism, making it difficult to extend to heterogeneous protocols and multi-task simulation scenarios. Another approach is the integration of CybORG and Mininet, which trains reinforcement learning agents in the CybORG abstract environment, translates their actions into real Mininet commands, captures the standard output stream, parses it into structured states, and returns them to the agent, thus replacing abstract simulation feedback with real container execution results. Essentially, this approach is geared towards specific network security scenarios and still lacks a unified policy interface for simulation internal parameters and simulation operation control.

[0007] In summary, among existing technologies for real-time network simulation and AI interaction, AI-driven real-time network simulation utilizes AI technology to automatically configure the network simulation environment. For example, Zhangba Network Security's AI intelligent platform can automatically complete the construction and configuration of network topology, targets, and vulnerabilities based on user needs and rules, by calling upon intelligence and vulnerability databases. Simultaneously, AI can be used to optimize network simulation models, enabling them to more accurately simulate real network behavior. Furthermore, it can perform intelligent analysis and prediction based on simulation results, providing decision support for network design, management, and optimization.

[0008] Real-time network simulation supports AI training and applications: It provides AI with realistic training environments and data. For example, Matrix-Game 2.0 generates a large amount of video data with detailed interactive annotations by building a scalable data production pipeline based on a game engine, which is used to train the world model. At the application level, real-time network simulation provides AI with a real-time interactive testing platform to verify its performance and effectiveness in different network scenarios.

[0009] Existing real-time simulation and AI interaction systems mostly focus on intelligent optimization at the level of specific tasks or protocols, lacking an architecture that can achieve intelligent perception and decision feedback through a unified interface for general-purpose simulation systems. To address this issue, this invention proposes an asynchronous intelligent control method for real-time simulation systems based on EMANE (Extendable Mobile Ad-hoc Network Emulator), an open-source distributed simulation framework that provides wireless network experimenters with a highly flexible modular environment for designing, developing, and testing simple and complex network architectures. This method achieves real-time perception and closed-loop control of the simulation state. Compared with existing technologies, this method can achieve closed-loop intelligent control of the network simulation state while ensuring simulation realism and real-time performance, thus significantly improving the dynamic response capability and intelligence level of the simulation system. This provides an effective means for research on real-time intelligent control of high-fidelity, large-scale networks. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of existing intelligent control methods for network simulation in terms of real-time performance, dynamic adaptability, and versatility, and to provide an asynchronous intelligent control method for real-time simulation systems. This method enables asynchronous acquisition of network states, intelligent decision-making, and dynamic feedback control while ensuring simulation fidelity, and supports continuous optimization and adaptive operation of the simulation process.

[0011] To achieve the above objectives, the technical solution of the present invention is an asynchronous intelligent control method for real-time simulation systems, which comprises three main parts: a simulation environment construction module, an asynchronous feedback acquisition module, and an intelligent control decision module.

[0012] Simulation Environment Construction Module: This module builds the network topology based on the real-time simulation platform EMANE, loads node, channel, and model configuration files, and defines simulation runtime parameters. The module supports node start / stop and parameter adjustment operations, providing an executable interface for subsequent intelligent control.

[0013] Asynchronous feedback acquisition module: During simulation, the system periodically collects key indicators such as node status, link quality, throughput, and latency through probe modules, and writes them to the status buffer asynchronously. The probe sampling process runs independently of the main simulation thread, ensuring that the simulation process is not affected by blocking. The collected data, after preprocessing, serves as the status input for the intelligent control decision module.

[0014] The intelligent control decision-making module utilizes reinforcement learning or deep reinforcement learning algorithms to analyze and reason about asynchronously acquired states (such as node traffic load, link quality, channel interference power, latency jitter, etc.). The agent generates action decisions based on the current state, outputting control commands, including node parameter adjustments, interface control, power configuration, or traffic scheduling, to the environment construction module. The decision-making process can continue during simulation without pausing the simulation or reducing fidelity. The specific adjustment values ​​are automatically generated by the learning algorithm during operation and updated in real time based on environmental feedback to form a convergent optimized control closed loop.

[0015] Based on the above architecture, this invention constitutes a complete feedback learning closed loop, and the specific steps are as follows:

[0016] 1) The simulation control system loads the network topology configuration and node model, and starts the real-time simulation platform. The system simultaneously starts the asynchronous acquisition thread and the intelligent decision-making thread to provide the operating environment for subsequent closed-loop control.

[0017] 2) The probe module periodically collects network status information as an independent process, including parameters such as link quality, node throughput, packet loss rate, and signal-to-noise ratio, and performs preprocessing and normalization. The sampled data from each network node is uniformly collected to the host-side proxy, and the TCP port and corresponding protocol rules are opened for the intelligent control module to read asynchronously, and used as the status input of the intelligent module.

[0018] 3) The reinforcement learning agent, based on the current state vector input and combined with historical experience and policy network weights, outputs action decisions. Actions may include control behaviors such as adjusting node transmit power, allocating link priorities, selecting channels, or modifying routing paths.

[0019] 4) The system control interface sends the control commands generated by the intelligent agent to the simulation platform. The platform dynamically updates relevant node parameters without interrupting the simulation, enabling real-time policy injection and execution.

[0020] 5) After control execution, the new network state is captured in the next sampling cycle. The system calculates the immediate reward after execution (such as reduced latency, increased throughput, reduced energy consumption, etc.) and returns it to the agent. The agent updates the policy network weights based on the feedback signal, achieving continuous learning and performance optimization.

[0021] The above process continuously loops during simulation, forming an adaptive control closed loop of simulation-data acquisition-decision-execution-feedback. As training progresses, the agent gradually learns the optimal control strategy, enabling the system to achieve dynamic optimization under different scenarios. This invention's method is based on a general wireless network simulation architecture. By introducing an intelligent control strategy, it asynchronously interacts the dynamic state of the simulated network with the intelligent control agent, achieving intelligent scheduling of node and link resources in a real-time simulation environment. Through asynchronous strategy updates, the intelligent control agent can adjust simulation parameters promptly when the simulated network changes, achieving real-time optimized control of complex network scenarios, thereby improving the real-time performance, adaptability, and decision-making capabilities of the network simulation system.

[0022] Compared with existing technologies, this invention has the following advantages: it breaks through the time coupling limitation of event-driven simulation frameworks, enabling continuous optimization without pausing the simulation; it improves the real-time response capability of the simulation system, allowing control strategies to be adjusted in real time according to changes in network status, achieving dynamic perception and instant control of multi-node, multi-task networks; it has good versatility and scalability, adapting to various network scenarios and protocol stack models, and can be extended to simulation networks of different scales and types by modifying sampling indicators and control interfaces, overcoming the shortcomings of existing simulation platforms in lacking proactive decision-making. Attached Figure Description

[0023] Figure 1 The overall architecture for asynchronous intelligent control methods for real-time simulation systems;

[0024] Figure 2 A flowchart for starting the simulation in the simulation environment construction module;

[0025] Figure 3 This is a flowchart illustrating the data acquisition part of the asynchronous feedback acquisition module.

[0026] Figure 4 A flowchart illustrating the process of issuing control commands to the asynchronous feedback acquisition module;

[0027] Figure 5 This is the algorithm logic diagram for the intelligent decision-making module. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0029] The system of this invention consists of the following functional modules: a simulation environment construction module, an asynchronous feedback acquisition module, and an intelligent control decision-making module. Figure 1 It illustrates the relationship between the three modules.

[0030] like Figure 2As shown, the simulation environment construction module automatically builds the network simulation runtime environment based on the overall simulation configuration file, providing underlying runtime support for asynchronous feedback acquisition and intelligent decision control. This enables subsequent policy injection, data sampling, and real-time feedback to interact via standardized interfaces. The simulation runtime environment adopts a host-side aggregation + containerized node deployment mode. Specifically, a unified virtual switching plane is created on physical / virtual hosts, and all simulation node containers connect their data interfaces to this switching plane; all simulation nodes can use common containerization technologies such as LXC and Docker. The logical connectivity and topological semantics between nodes are determined by the EMANE simulation daemon running within each container based on the physical layer / link layer model. The specific steps include the following:

[0031] 1) The system reads the overall simulation configuration file or script (e.g., an environment definition file based on XML or JSON format), and parses the number of nodes, link topology, model parameters, and runtime environment configuration. Based on the configuration content, the system automatically generates configuration files for each container.

[0032] 2) The system creates one or more virtual bridges (e.g., br-sim0) on the simulation host and assigns a peer-to-peer virtual Ethernet interface (veth) to each simulation node container. Each container's veth interface enters its network namespace, and the other end connects to the unified bridge on the host side, thus forming a fully connected physical switching plane at Layer 2. The host does not perform service-level drop or filtering of traffic between nodes, ensuring that all containers are reachable at the link layer.

[0033] 3) During container startup, the system dynamically loads the corresponding wireless ad hoc network protocol or channel modeling module according to the scenario configuration, such as loading the OLSR or BATMAN-adv protocol stack module to simulate link layer routing and neighbor discovery.

[0034] 4) The EMANE daemon is started within the container. This daemon calculates the reachability and reception probability of each data frame based on the environmental scenario configuration (transmit power, path loss model, node location, etc.) and decides whether to pass the received data to the upper-layer protocol stack or discard it. Thus, the daemon kernel selectively discards packets that are unreachable due to path loss or interference, rather than statically isolating them by the host bridge.

[0035] 5) When the intelligent control module needs to adjust the topology or link quality, it does not directly modify the host bridge or delete the veth interface. Instead, it sends parameter modification commands (such as transmit power, path loss, routing algorithm metrics, etc.) to the simulation kernel inside the container through a standardized control interface. Upon receiving the command, the simulation kernel immediately adjusts its link decision logic, thereby achieving dynamic topology reconstruction without altering the host network structure.

[0036] 6) After the simulation environment is built, the data acquisition agent deployed inside each container periodically reports link layer and protocol layer metrics (such as SNR, reception success rate, queue length, routing next-hop information, etc.) to the central Broker. Nodes that generate or inject traffic can also obtain the corresponding data. The intelligent control module, based on the state vector generation strategy provided by the Broker, sends the data to the simulation kernel through a standardized control interface. After execution, the simulation kernel generates new link behaviors and returns feedback in the next sampling cycle, forming a closed-loop control system of simulation—acquisition—decision—execution—feedback.

[0037] Through this process, the simulation environment achieves fully automated deployment from topology definition to node operation, forming a schedulable simulation platform that can be controlled in real time by upper-level intelligent control strategies.

[0038] The asynchronous feedback acquisition module establishes a highly reliable, loosely coupled data acquisition and command interaction mechanism between simulation nodes and the control center, enabling asynchronous aggregation and online feedback of simulation status information. Its core mechanism is built upon the ZeroMQ communication framework and the Protobuf serialization protocol, employing a publish / subscribe pattern to collect and aggregate simulation node data. Figure 3 The process of data collection and aggregation was demonstrated, specifically including the following steps:

[0039] 1) Each simulation node starts an independent data acquisition agent process to monitor and collect real-time metrics of the simulation processes within that node (such as the EMANE daemon, traffic generator, etc.), including physical layer link quality, RF parameters, MAC layer packet statistics, routing status, and traffic information. The data acquisition agent starts based on a configuration file and listens on the node's discovery and publish ports. The discovery port is used for probe registration and availability management, while the publish port is used to asynchronously publish node data in the form of a message stream.

[0040] 2) The master control unit starts the Broker proxy service process and registers the discovery / publish addresses within each container node according to the configuration file. The Broker acts as a data relay center, listening on a unified discovery port and data publishing port. Internally, the Broker exchanges messages with processing threads via ZeroMQ's PAIR communication sockets. Data published by each node is serialized using Protobuf and then aggregated into a unified data stream by the Broker. External monitoring modules or intelligent decision-making modules can subscribe to this port to receive global status data in real time.

[0041] 3) The data acquisition agent periodically publishes internal node probe data at configured intervals (e.g., 100ms to 1s). The broker maintains a unified timestamp index and node ID mapping table, enabling time alignment and dynamic packet loss compensation for asynchronous data from multiple nodes. For metrics with high real-time requirements (such as SINR or packet loss rate), the system allows enabling a fast sampling mode and storing the most recent window data through a circular buffer queue, providing status statistics within a sliding time window for the upper-layer intelligent decision-making module.

[0042] In addition to the data acquisition channel, an independent control channel is also needed to realize the dynamic management and policy distribution of each simulation node, forming an intelligent decision-making underlying control system for data reporting, policy distribution, and execution feedback. Figure 4 The process is shown, and the specific implementation is as follows:

[0043] 1) The control channel uses an independent TCP server deployed on the main control unit, decoupled from the data publishing channel. The main control unit maintains a control dispatch agent (Control Dispatcher), which listens on a predefined port and is responsible for receiving control requests from the upper-layer intelligent decision-making module and forwarding them to the target nodes. Each simulation node corresponds to a control agent (ControlAgent), which is permanently resident and listens for policy instructions from the main control unit.

[0044] 2) Control messages are transmitted via TCP streams and decoded and distributed by a parser at the node. After parsing, commands are mapped to specific execution methods, such as modifying configuration files, updating runtime parameters, or calling underlying system commands.

[0045] 3) After execution, the control agent sends the execution result (success / failure status and returned data) back to the master control unit via TCP. The intelligent decision-making module receives this data and updates the policy accordingly. Combined with real-time collected network performance indicators (such as link quality, throughput, and latency), a complete intelligent scheduling closed loop is formed.

[0046] Control messages are encapsulated in protobuf format and transmitted via length prefix framing. Each message includes a 4-byte network sequence length field to identify message boundaries and ensure accurate unpacking in a continuous TCP stream. Through protobuf messages, the control channel implements two basic commands: Query (querying configuration, statistics, etc.) and Update (modifying configuration). Taking sending a command to the EMANE control process as an example, the parser constructs the corresponding message and sends the command to modify the configuration. Some parameters and functions of the command are as follows:

[0047] The Query command is used to query the running configuration, statistics, or link status of the current node's components. For example:

[0048] type = TYPE_REQUEST_QUERY

[0049] query.configuration.buildId=3

[0050] query.configuration.names=["txpower","pathlossmode"]

[0051] After receiving the data, the node maps it to a query operation through the parser and requests the EMANE daemon to return the corresponding configuration items and statistical values, which are then reported to the decision-making module.

[0052] The Update command is used to modify node operating parameters or clear statistics in real time. For example:

[0053] type = TYPE_REQUEST_UPDATE

[0054] update.configuration.buildId=3

[0055] update.configuration.parameters.name="txpower"

[0056] update.configuration.parameters.values.u32Value=20

[0057] After the node is resolved, it calls the local emane control interface and sends parameter update requests to the daemon process through the EMANE Remote Control Port to dynamically adjust indicators such as transmit power and channel model.

[0058] The intelligent control decision module is responsible for analyzing, reasoning, and generating control strategies from the network status data provided by the asynchronous acquisition module, thereby enabling dynamic adjustment and adaptive optimization of the simulation process. Figure 5 This demonstrates the main workflow of the module. Running as an independent process, it interacts with the simulation platform through a standardized interface, independent of the simulation main thread's time stepping, thus achieving asynchronous control closed loop. The main steps include:

[0059] 1) The module receives node and link status vectors from the asynchronous feedback acquisition module, including metrics such as node throughput, packet loss rate, signal-to-noise ratio, latency, and queue length. Input data is received in real time via the ZeroMQ subscription mechanism and, after normalization and feature compression, is sent to the intelligent decision-making logic.

[0060] 2) The intelligent decision-making logic can be implemented using reinforcement learning, rule-based control, or a hybrid algorithm structure. The agent calculates and outputs the optimal set of actions based on the current state and historical experience. Action types include, but are not limited to: node transmit power adjustment; channel selection and multiplexing control; route priority or congestion threshold adjustment; link load balancing and traffic migration. This process can continue during simulation, and the agent's decision-making cycle can be dynamically adjusted according to computing resources to ensure a balance between control real-time performance and simulation fidelity.

[0061] 3) The decision module serializes the generated control commands using Protobuf and then publishes them to the simulation control port via ZeroMQ. Each command includes the target node number, operation type, parameter value, and valid timestamp. Upon receiving the command, the simulation platform immediately executes the corresponding parameter modification operation.

[0062] The simulation platform returns the new link state in the next sampling period, and the agent calculates the reward R based on the performance change. t The system updates strategy parameters accordingly. Under different channel states, network loads, and topology conditions, the system can dynamically optimize key performance indicators, achieving network adaptability and convergent near-optimal performance. While maintaining network stability, it achieves asynchronous closed-loop optimization of simulation, data acquisition, decision-making, execution, and feedback.

[0063] After the decision is executed, the operating state of the simulation node changes accordingly. New network performance metrics are captured in the next round of sampling and returned to the agent. The system calculates immediate reward values ​​(such as reduced latency, increased throughput, or reduced energy consumption), and the agent updates its policy weights accordingly to achieve continuous optimization.

[0064] The following example uses a wireless multi-hop ad hoc network simulation. The system utilizes an intelligent control decision module to achieve dynamic adaptive adjustment of node transmit power. This process is based on asynchronously acquired network state information, generates control commands through agent inference, and executes them in real time on the simulation platform, thereby optimizing overall link quality and network throughput performance. The specific steps are as follows:

[0065] 1) In each control cycle t, the acquisition module obtains the inter-node communication indicators from the simulation platform to form a state vector: S t =[SINR ij (t),Delay ij (t),Throughput i (t),QueueLen i (t)]where SINR ij The signal-to-noise ratio between node i and node j; Delay ij The average link delay between node i and node j; Throughput i Node i throughput; QueueLeni The queue length is sent to node i. The state data, after normalization and moving average processing, is input into the agent for policy reasoning.

[0066] 2) The intelligent control module is modeled based on a reinforcement learning framework, mapping the system state to control actions: a t =π θ (s t ) where π θ Let θ represent the policy function. The action space is defined as the set of node transmit power adjustments: a t ∈{ΔP i |ΔP i ∈[-δ,+δ]}. ΔP i Defined by the system parameter configuration file, it can be based on the node's transmit power limit P. max The noise estimate is dynamically adjusted, typically within a range of ±1 to 3 dB. Each adjustment takes effect immediately via the EMANE control interface, and link metrics are recalculated in the next sampling period. When the link signal-to-noise ratio decreases or latency increases, the agent outputs a positive power adjustment to enhance link quality; conversely, it moderately reduces transmit power to decrease interference and energy consumption. The agent updates its strategy based on the principle of maximizing expected return. Where R t The reward value for the current period is defined as the system performance gain function: R t =w1·ΔSINR avg -w2·ΔDelay avg -w3·ΔP total The weights w1, w2, and w3 are determined by the task requirements and are used to balance communication quality and energy consumption goals.

[0067] 3) To improve the convergence speed and stability of the strategy, the system can adopt a hierarchical control mechanism. The upper-level policy network is responsible for outputting the global control trend (such as the direction of power increase / decrease), while the lower-level local controllers perform fine-grained adjustments based on real-time feedback from nodes, avoiding global policy oscillations. A higher exploration rate ε is used in the initial stage to improve adaptability to multiple scenarios; as the average reward stabilizes, it is gradually reduced to achieve stable convergence.

[0068] 4) After generating a control action, the agent sends an Update command to the target node via the control channel, including the target node number, operation type, and parameter values. The node-side control agent receives the message, parses and executes it, updating the MAC or PHY layer parameters by calling the message interface in the EMANERemote Control API, thus achieving real-time power adjustment. Parameter modifications take effect immediately without pausing the main simulation thread, thereby maintaining simulation continuity and control real-time performance. This mechanism can achieve asynchronous closed-loop optimization of simulation—acquisition—decision—execution—feedback while maintaining network stability.

[0069] 5) As training progresses, the agent gradually converges to a near-optimal control strategy, thereby achieving adaptive optimization of the intended goal: With iterative advancement, the agent gradually forms a stable policy distribution under different channel states, interference environments, and topology conditions. When the policy gradient update magnitude is lower than a set threshold, the system considers the model to have approached the optimal policy, manifested as an increase in average SINR, a decrease in average latency, and a reduction in total power consumption.

[0070] It should be noted that the intelligent decision-making method described in this embodiment uses a reinforcement learning framework as an example. Its core idea is to construct a feedback loop by asynchronously acquiring states to achieve adaptive optimization. In practical applications, it can also be implemented based on deep neural networks, adaptive PID, genetic algorithms, or other optimization methods; the action types are not limited to power control, but can also be extended to multi-dimensional control tasks such as channel allocation, routing selection, queue scheduling, and access control.

[0071] The examples described above are merely preferred embodiments of the present invention, and while their descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for asynchronous intelligent control of a real-time simulation system, characterized in that Based on the architecture, the simulation environment construction module, the asynchronous feedback acquisition module and the intelligent control decision module are connected; The simulation environment construction module: This module is based on the real-time simulation platform EMANE to build network topology, load node, channel and model configuration file, and define simulation running parameters. The module supports node start, stop and parameter adjustment operations, and provides an executable interface for subsequent intelligent control; The asynchronous feedback acquisition module: During the simulation running process, the system periodically acquires node state, link quality, throughput rate, and delay key indicators through the probe module, and writes them into the state cache in an asynchronous manner; The probe sampling process runs independently of the simulation main thread to ensure that the simulation process is not affected by blocking; The collected data, after preprocessing, are used as the state input of the intelligent control decision module; The intelligent control decision module: Reinforcement learning or deep reinforcement learning algorithms are used to analyze and infer the state of the asynchronous acquisition; The agent generates action decisions based on the current state, and outputs control instructions including node parameter adjustment, interface control, power configuration or traffic scheduling to the environment construction module; The decision-making process can continue during the simulation running without pausing the simulation or reducing the fidelity, achieving dynamic adaptive optimization; Based on the above architecture, a complete feedback learning closed loop is formed, and the specific steps are as follows: 1) The simulation control system loads the network topology configuration and node model, and starts the real-time simulation platform; The system also starts the asynchronous acquisition thread and the intelligent decision thread to provide an operating environment for subsequent closed-loop control; 2) The probe module periodically acquires network state information, including link quality, node throughput, packet loss rate, signal-to-noise ratio, etc., and performs preprocessing and normalization; The sampling data of each network node are collected to the host-side agent, which opens a tcp port and corresponding protocol rules for the intelligent control module to read asynchronously; 3) The reinforcement learning agent outputs action decisions based on the current state vector input, combined with historical experience and policy network weights; The actions can include node transmit power adjustment, link priority allocation, channel selection or routing path modification, etc. 4) The system control interface sends the control instructions generated by the agent to the simulation platform; The platform dynamically updates the relevant node parameters without interrupting the simulation running, realizing real-time policy injection and execution; 5) After the control execution, the new network state is captured in the next period of sampling; The system calculates the immediate reward after execution, including delay reduction, throughput rate improvement, energy consumption reduction, and returns it to the agent; The agent updates the policy network weights based on the feedback signal to achieve continuous learning and performance optimization; The above process continues to circulate during the simulation running, forming a self-adaptive control closed loop of simulation-acquisition-decision-execution-feedback.

2. The real-time simulation system-oriented asynchronous intelligent control method according to claim 1, characterized in that, The simulation environment construction module automatically builds the network simulation running environment based on the simulation overall configuration file, provides bottom layer running support for asynchronous feedback acquisition and intelligent decision control, and enables subsequent policy injection, data sampling and real-time feedback to interact through standardized interfaces; The specific steps are as follows: 1) The system reads the simulation overall configuration file or script, parses the number of nodes, link topology, model parameters and runtime environment configuration therein; according to the configuration content, the system automatically generates the configuration file corresponding to each container, and then establishes the containerized simulation running environment, including instantiating an independent container running space for each node, configuring the necessary virtual network interface, process management parameter and resource quota; 2) The module automatically creates a simulation bridge according to the topology description, and configures the network namespace, interface binding and IP address allocation of each node through the system script; during the simulation running process, the control instructions issued by the intelligent decision module are used to reload or adjust part of the topology relationship; 3) After the simulation environment is built, the system automatically deploys a local data collection agent and a remote control interface for each node; The data collection agent is responsible for reading the real-time state information and routing state of the link layer from the internal simulation process and container network configuration of the node, including throughput, signal-to-noise ratio, routing next hop, and if there is a node generating and injecting traffic, the corresponding data is also obtained; the control interface exposes the node parameter modification entry in the form of a standardized API, including power adjustment, protocol parameter change, routing algorithm switching; 4) After the configuration and interface initialization are completed, the system calls the node management script to start the EMANE simulation process of each node, initializes the model layer, event layer and wireless channel module; After all the nodes are started, the system registers them to the global simulation control scheduler to support unified task management, state monitoring and control instruction distribution; Through this process, the simulation environment realizes the full automation of deployment from topology definition to node running, forming a schedulable simulation platform that can be real-time controlled by the upper intelligent control strategy.

3. The real-time simulation system-oriented asynchronous intelligent control method according to claim 1, characterized in that, The asynchronous feedback collection module is used to establish a reliable and low-coupling data collection and instruction interaction mechanism between the simulation node and the control center, realize the asynchronous aggregation and online feedback of the simulation state information; the core mechanism is built based on the ZeroMQ communication framework and the Protobuf serialization protocol, and adopts the Publish / Subscribe mode to collect and aggregate the simulation node data, including the following steps: 1) An independent collection agent process is started in each simulation node, which is used to monitor and collect the internal simulation process of the node, including the real-time indicators of the EMANE daemon process and the traffic generator, such as physical layer link quality, radio frequency parameters, MAC layer packet statistics, routing state and traffic information, etc.; the collection agent is started based on the configuration file, and listens to the discovery port and the publish port in the node; wherein the discovery port is used for probe registration and availability management, and the publish port is used for asynchronously publishing node data in the form of message stream; 2) Master starts Broker agent service process, and registers discovery / publish address in each container node according to configuration file; Broker acts as a data transfer center, and listens to unified discovery port and data publishing port; Broker exchanges messages with processing thread through PAIR communication socket of ZeroMQ; data published by each node is serialized by Protobuf, and is gathered by Broker into data stream in unified format; external monitoring module or intelligent decision module can receive global state data in real time by subscribing to the port; 3) Acquisition agent publishes internal probe data at a configured period; Broker maintains unified timestamp index and node ID mapping table, and can realize time alignment and dynamic packet loss compensation of asynchronous data of multiple nodes; For indicators with high real-time requirements, including SINR or packet loss rate, the system allows to enable fast sampling mode, and stores recent window data through ring buffer queue, so as to provide state statistics in sliding time window for upper intelligent decision module.

4. The real-time simulation system-oriented asynchronous intelligent control method according to claim 1, characterized in that, In addition to data acquisition channel, an independent control channel is provided to realize dynamic management and policy issuing of each simulation node, and intelligent decision bottom control of data reporting, policy issuing and execution feedback is formed; the specific implementation is as follows: 1) Control channel is deployed on master through independent TCP Server, and is decoupled from data publishing channel; master maintains a control dispatcher, listens to a predefined port, receives control request from upper intelligent decision module, and uniformly forwards the request to target node; each simulation node corresponds to a control agent, which constantly listens to policy instruction from master; 2) Control message is transmitted through TCP stream, and is decoded and distributed by parser on node side; after parsing, the command is mapped to specific execution for operation, including modifying configuration file, updating running parameter or calling underlying system command; taking EMANE control end as an example, Remote Control Port provides a standard TCP control port, and Query is realized through protobuf message, including querying configuration, statistical quantity and modifying configuration; parser constructs corresponding message, and sends instruction of modifying configuration to EMANE daemon process.

5. The real-time simulation system oriented asynchronous intelligent control method according to claim 1, characterized in that, Intelligent control decision module is responsible for analyzing, reasoning and generating control strategy based on network state data provided by asynchronous acquisition module, and realizes dynamic adjustment and adaptive optimization of simulation process; working process; the module runs as an independent process, interacts with simulation platform through standardized interface, and does not depend on time stepping of simulation main thread, so as to realize asynchronous control closed loop; including the following steps: 1) Module receives node and link state vector from asynchronous feedback acquisition module, and the content includes node throughput, packet loss rate, signal-to-noise ratio, delay and queue length indicators; input data is accessed in real time through ZeroMQ subscription mechanism, and is sent to intelligent decision logic after normalization and feature compression; 2) The intelligent decision logic is implemented by reinforcement learning, rule control or hybrid algorithm structure; the agent calculates and outputs the optimal action set according to the current state and historical experience; The action types include: node transmit power adjustment; channel selection and multiplexing control; routing priority or congestion threshold adjustment; link load balancing and traffic migration; the process can be continuously performed during the simulation running, and the decision period of the agent can be dynamically adjusted according to the computing resources to ensure the balance between control real-time performance and simulation fidelity; 3) The decision module serializes the generated control instructions through Protobuf and publishes them to the simulation control port through ZeroMQ; each instruction includes the target node number, operation type, parameter value and effective timestamp, and the simulation platform receives and immediately executes the corresponding parameter modification operation; After the decision is executed, the running state of the simulation node changes; the new network performance indicators are captured in the next round of sampling and returned to the agent, and the system calculates the immediate reward value, including delay reduction, throughput improvement or energy consumption reduction, and the agent updates the policy weight accordingly to achieve continuous optimization.

6. The real-time simulation system oriented asynchronous intelligent control method according to claim 1, characterized in that, The system uses the intelligent control decision module to realize the dynamic adaptive adjustment of node transmit power; this process is based on the asynchronously collected network state information, and the control instructions are generated by the agent reasoning, which optimizes the overall link quality and network throughput performance; the specific steps are as follows 1) In each control period t, the acquisition module obtains the communication indicators between nodes from the simulation platform to form the state vector: s t = [SINR ij (t), Delay ij (t), Throughput i (t), QueueLen i (t)] where SINR ij is the signal-to-noise ratio between node i and node j; Delay ij is the average delay of the link between node i and node j; Throughput i is the throughput of node i; and QueueLen i is the length of the sending queue of node i; after the state data is normalized and sliding average processed, it is input to the agent for policy inference; 2) The intelligent control module is modeled based on a reinforcement learning framework, mapping system states to control actions: a t = π θ (s t ) where π θ represents a policy function represented by parameters θ; the action space is defined as a set of node transmit power adjustments: a t ∈{ΔP i |ΔP i ∈[-δ,+δ]} When the link signal-to-noise ratio decreases or the delay increases, the intelligent agent will output a positive power adjustment to enhance the link quality; otherwise, it will moderately reduce the transmit power to reduce interference and energy consumption; the intelligent agent updates the policy according to the principle of maximizing expected returns: where R t is the reward value of the current period, defined as the system performance gain function: R t = w1·ΔSINR avg -w2·ΔDelay avg -w3·ΔP total , the weights w1, w2, w3 are determined by the task requirements, used to balance the communication quality and energy consumption goals; 3) After the agent generates the control action, it sends the Update instruction to the target node through the control channel, including the target node number, operation type, parameter value and effective timestamp; the node end control agent receives and parses the message for execution, updates the MAC or PHY layer parameters by calling the message interface in the EMANE Remote Control API, realizes real-time power adjustment; the parameter modification takes effect immediately, without the need to pause the simulation main thread, thus maintaining the simulation continuity and control real-time performance.

7. The asynchronous intelligent control method for real-time simulation system according to claim 1, characterized in that, The simulation platform returns a new link state in the next sampling period, and the agent calculates the reward R according to the performance change t , and updates the policy parameters; after multiple iterations, the agent gradually learns the optimal power control strategy under different channel states, interference environments and topological structures, and realizes the adaptive evolution of network performance; while maintaining network stability, the asynchronous closed-loop optimization of simulation-collection-decision-execution-feedback is realized.

8. The real-time simulation system oriented asynchronous intelligent control method according to claim 1, characterized in that, As the training progresses, the agent gradually converges to a near-optimal control strategy, thereby achieving adaptive optimization of the proposed target.

Images