Network attack detection method and device based on deep reinforcement learning and topology reconstruction

Abstract

The embodiment of the invention relates to the field of network security, and provides a network attack detection method and device based on deep reinforcement learning and topology reconstruction, and the method comprises the steps: collecting network flow data, node state information and security logs in real time, carrying out the dynamic modeling of a connection relation and communication behaviors between nodes, and generating a dynamic attack graph; analyzing the current network state of the dynamic attack graph through a deep reinforcement learning agent to obtain an optimal defense action; generating a strategy instruction based on the optimal defense action, and sending the strategy instruction to a network strategy execution point; by calling an application programming interface of a strategy execution point, blocking of malicious IPs, isolation of infected nodes or redirection of malicious traffic is dynamically executed, network topology reconstruction is triggered to rapidly repair or isolate fault nodes, and instant detection and interception of attack behaviors are achieved. According to the embodiment of the invention, active perception and intervention are carried out on attack behaviors, and intelligent decision and dynamic self-healing of network topology are realized through deep reinforcement learning.

Description

Technical Field

[0001] This application relates to the field of network security, and more specifically to a network attack detection method and apparatus based on deep reinforcement learning and topology reconstruction. Background Technology

[0002] With the development of information technology and the continuous expansion of network scale, network security protection has become a core component in ensuring the stable operation of critical infrastructure. Current network attack detection technologies mainly rely on signature-based intrusion detection systems (such as signature matching tools like Snort) or traditional machine learning algorithms (such as classification models like support vector machines and random forests). In static network environments, such as small LANs or fixed-topology internal networks, these methods can achieve basic attack identification and response based on predefined attack signature databases or historical data, effectively addressing known threat types.

[0003] However, with the expansion of network scale and the increasing complexity of attack methods, existing technologies generally adopt a passive detection mode. This means that defense mechanisms are triggered only after actual damage has been caused by the attack, lacking real-time dynamic decision-making capabilities. This prevents proactive intervention and containment in the early stages of an attack (such as the initial propagation phase), leading to delayed security responses, expanded attack spread, and irreversible losses such as data leaks and service interruptions. Simultaneously, network topologies are typically designed to be static and fixed, lacking adaptive adjustment capabilities. When an attack is detected, network connections cannot be quickly reconstructed to cut off the attack propagation path. For example, in lateral movement attacks, the communication links between infected nodes and other nodes cannot be blocked in real time, resulting in slow and limited effectiveness of defense measures. Especially in highly dynamic network scenarios, such as the frequent creation, migration, and destruction of virtual machines in cloud computing platforms, and the dynamic access and disconnection of numerous devices in IoT systems, network topologies exhibit high-frequency and highly random changes. Existing technologies are ill-suited to adapting to such real-time environmental changes, and are unable to effectively isolate infected nodes (e.g., in ransomware attacks, they cannot quickly identify and block the spread of malicious processes), nor can they repair faulty areas in a timely manner (e.g., in distributed denial-of-service attacks, they cannot dynamically optimize traffic routing to alleviate bandwidth bottlenecks). This results in a severe lack of network resilience, with security threats accumulating and creating a vicious cycle.

[0004] These shortcomings not only render traditional methods ineffective in dealing with advanced persistent threats (APTs) and zero-day attacks, but also further exacerbate the vulnerability of network systems, failing to meet the urgent needs of modern networks for proactive defense, adaptive protection, and persistent threat management. In conclusion, a novel technical solution is urgently needed to address at least one of the technical problems inherent in existing technologies. Summary of the Invention

[0005] This application provides a network attack detection method and apparatus based on deep reinforcement learning and topology reconstruction, which can actively perceive and intervene in attack behavior, thereby achieving intelligent decision-making and dynamic self-healing of network topology through deep reinforcement learning, solving the technical problems of slow response, rigid topology and insufficient resilience in the prior art.

[0006] In a first aspect, embodiments of this application provide a network attack detection method based on deep reinforcement learning and topology reconstruction, the method comprising: Real-time collection of network traffic data, node status information and security logs; dynamic modeling of the connection relationship and communication behavior between nodes; generation of dynamic attack graphs including node vulnerability status, real-time attack paths, attack propagation paths and real-time threat indicators. The dynamic attack graph is input into a deep reinforcement learning agent, which analyzes the current network state to obtain the optimal defense action. Based on the optimal defense action, policy instructions are generated and sent to network policy enforcement points; the policy enforcement points include firewalls, intrusion prevention systems, and software-defined network controllers. By calling the application programming interface of the policy execution point, the system can dynamically block malicious IPs, isolate infected nodes, or redirect malicious traffic, and trigger network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving real-time detection and interception of attack behaviors.

[0007] Secondly, embodiments of this application provide a network attack detection device based on deep reinforcement learning and topology reconstruction, which has the function of implementing the network attack detection method based on deep reinforcement learning and topology reconstruction provided in the first aspect above. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function, and the modules can be software and / or hardware. In one embodiment, the network attack detection device based on deep reinforcement learning and topology reconstruction includes: The modeling module is configured to collect network traffic data, node status information and security logs in real time, dynamically model the connection relationship and communication behavior between nodes, and generate a dynamic attack graph that includes node vulnerability status, real-time attack path, attack propagation path and real-time threat indicators. The decision module is configured to input the dynamic attack graph into a deep reinforcement learning agent, analyze the current network state through the deep reinforcement learning agent, obtain the optimal defense action, generate policy instructions based on the optimal defense action, and send them to the network policy execution point; the policy execution point includes firewalls, intrusion prevention systems, and software-defined network controllers. The execution module is configured to dynamically block malicious IPs, isolate infected nodes, or redirect malicious traffic by calling the application programming interface of the policy execution point, and trigger network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving real-time detection and interception of attack behaviors.

[0008] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the network attack detection method based on deep reinforcement learning and topology reconstruction as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the network attack detection method based on deep reinforcement learning and topology reconstruction described in the first aspect.

[0010] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to execute the network attack detection method based on deep reinforcement learning and topology reconstruction provided in the first aspect.

[0011] Compared to existing technologies, this application's embodiments achieve proactive threat perception by constructing a dynamic attack graph that integrates multi-source heterogeneous data. It also introduces a deep reinforcement learning agent for online policy optimization and decision-making, synchronously linking with a software-defined network controller to perform topology reconfiguration, thus constructing a closed-loop proactive defense system. Because this application's embodiments employ a technical path combining data-driven intelligent decision-making with dynamic network structure evolution, rather than the passive response mechanism relying on static feature library matching or fixed rule bases found in existing technologies, this application's embodiments can overcome the dual bottlenecks of traditional methods in response timeliness and topology adaptability, implementing precise intervention and path blocking at the initial stage of attack propagation. Therefore, the synergistic mechanism of dynamic attack graphs and deep reinforcement learning in this application's embodiments enables continuous environmental perception and policy self-evolution capabilities, dynamically generating optimal defense sequences for unknown threats. Because the dynamic attack graph integrates node vulnerability status, communication behavior time-series characteristics, and threat indicators in real time, and the deep reinforcement learning agent continuously improves decision-making quality through online experience replay and policy gradient optimization, the defense scheme obtained in this application has high timeliness, high adaptability, and high robustness. It can exert active immunity and resilience enhancement effects, and can effectively solve the core pain points of attack detection lag, topology rigidity, and insufficient system resilience in highly dynamic network environments. It provides continuous, adaptive, and verifiable security protection capabilities for key scenarios such as cloud computing and the Internet of Things, and reduces the risk of data leakage and service interruption. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a network attack detection method based on deep reinforcement learning and topology reconstruction according to an embodiment of this application. Figure 2 This is a schematic diagram of the network attack detection device based on deep reinforcement learning and topology reconstruction according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a mobile phone in one embodiment of this application; Figure 4 This is a schematic diagram of a server structure in one embodiment of this application. Detailed Implementation

[0013] This application provides a network attack detection method and apparatus based on deep reinforcement learning and topology reconstruction, which can be applied to intelligent defense systems in network security protection scenarios that require high dynamic adaptability and proactive defense capabilities. The intelligent defense system may include a data perception unit, a dynamic modeling unit, an intelligent decision-making unit, and a policy execution unit. The above units can be integrated and deployed on a unified security operation platform, or they can be deployed separately using a distributed architecture (such as deploying the data perception unit on a network probe and the decision-making unit on a cloud analysis cluster). The system comprises the following components: a data perception unit for real-time collection of network traffic data, node status information, and security logs; a dynamic modeling unit for integrating graph neural networks and time-series analysis techniques to dynamically model node connectivity and communication behavior, generating a dynamic attack graph that includes node vulnerability status, potential attack paths, propagation patterns, and multi-dimensional threat indicators; an intelligent decision-making unit for inputting the dynamic attack graph into a pre-trained and online fine-tuned deep reinforcement learning agent (e.g., based on the PPO / SAC algorithm), outputting in real-time the optimal defense action sequence that balances blocking effectiveness and business continuity, and generating fine-grained policy instructions; and a policy execution unit for accurately distributing instructions to policy execution points such as firewalls, intrusion prevention systems (IPS), and software-defined networking (SDN) controllers, and collaboratively executing traffic blocking, node isolation, traffic redirection, and topology reconstruction operations through standardized APIs (e.g., RESTful, OpenFlow). The data perception unit can be a traffic acquisition agent or a lightweight probe program deployed on the mirror port of a switch; the dynamic modeling unit can be a graph computing service module running on a security analysis server; the intelligent decision-making unit can be an inference engine that encapsulates a deep reinforcement learning model (supporting TensorRT acceleration); and the policy execution unit can be an automated orchestration component integrated into the SOAR platform, supporting closed-loop optimization of policy effect feedback.

[0014] In practical applications, this application also provides a network attack detection device based on deep reinforcement learning and topology reconstruction. The device includes: a modeling module configured to collect network traffic data, node status information, and security logs in real time; dynamically model the connection relationships and communication behaviors between nodes; and generate a dynamic attack graph containing node vulnerability status, real-time attack paths, attack propagation paths, and real-time threat indicators. A decision-making module configured to input the dynamic attack graph into a deep reinforcement learning agent; analyze the current network state through the deep reinforcement learning agent to obtain the optimal defense action; generate policy instructions based on the optimal defense action; and send them to network policy execution points; the policy execution points include firewalls, intrusion prevention systems, and software-defined network controllers. An execution module configured to dynamically execute actions such as blocking malicious IPs, isolating infected nodes, or redirecting malicious traffic by calling the application programming interface of the policy execution points; and triggering network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving immediate detection and interception of attack behaviors.

[0015] The solutions provided in this application relate to the interdisciplinary fields of network security and artificial intelligence, focusing on solving core problems such as delayed attack detection, passive defense, and topology rigidity in highly dynamic network environments. The specific solutions are illustrated in the following embodiments.

[0016] In existing technologies, network attack detection primarily employs signature-based intrusion detection systems (such as Snort) or traditional machine learning models trained on historical data (such as SVM and random forests). These technologies mainly achieve efficient matching through pre-built databases of known attack signatures or utilize offline-trained classifiers for post-incident traffic analysis to balance detection accuracy and computational overhead. However, the detection mechanisms of existing technologies exhibit a reactive nature, typically triggering alerts after observable traces of attack behavior emerge, failing to proactively intervene at the initial stages of the attack chain (such as the initial intrusion point), resulting in a delayed response window. The fixed network topology and the strong binding of defense strategies to physical / logical connections mean that when faced with dynamic attacks such as lateral movement and ransomware propagation, there is a lack of real-time communication path reconstruction capabilities, making it difficult to block attack spread. Furthermore, in highly dynamic scenarios such as elastic scaling of cloud computing resources and high-frequency access of IoT devices, static signature databases are prone to failure, traditional models lack generalization capabilities, and are almost insensitive to unknown threats such as zero-day attacks and APTs. Moreover, policy adjustments rely on manual intervention, severely weakening network resilience.

[0017] Compared to existing technologies, in this application embodiment, a proactive threat situation perception is achieved by constructing a dynamic attack graph that integrates multi-source heterogeneous data, and a deep reinforcement learning agent is introduced for online strategy optimization and decision-making, synchronously linking the software-defined network controller to perform topology reconstruction operations, thus constructing a closed-loop proactive defense system.

[0018] In some implementations, the modeling module, decision-making module, and execution module are deployed in a separate, distributed manner. The network attack detection system based on deep reinforcement learning and topology reconstruction provided in this application includes a lightweight probe cluster (corresponding to the modeling module) deployed at the network edge and a security intelligent hub (integrating the decision-making and execution modules) deployed in the cloud. The probe cluster collects switch mirror traffic, host agent status logs, and security device alarms in real time, and uploads them to the security intelligent hub via a TLS encrypted channel after de-identification and compression. The hub uses a graph computing engine to generate a dynamic attack graph and inputs it into a deep reinforcement learning inference engine to generate defense policy instructions. After digital signature verification, the instructions are sent to policy execution units (including SDN controllers and cloud firewall clusters) distributed in the network core layer, which collaboratively execute operations such as logical isolation of infected nodes, redirection of malicious traffic to honeypots, and topology reconstruction of critical business links. Simultaneously, the execution results (such as changes in traffic after isolation and business SLA metrics) are fed back to the intelligent hub in real time, driving the reinforcement learning model to fine-tune online and continuously evolve the closed loop.

[0019] It should be noted that the servers involved in the embodiments of this application can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers or edge computing nodes that provide capabilities such as cloud security services, big data analysis platforms, AI inference services, and SDN control planes. The terminal devices involved in the embodiments of this application refer to dedicated security devices with data collection and policy execution capabilities deployed in critical network locations, including but not limited to network traffic probes, host security agents, SDN switches, smart gateways, and IoT security gateways. They may also include industrial control terminals with edge computing capabilities, 5G UPF devices, etc., used to achieve a layered protection system that combines rapid local response with cloud-based collaborative defense.

[0020] Reference Figure 1 , Figure 1 This is a flowchart illustrating a network attack detection method based on deep reinforcement learning and topology reconstruction, provided as an embodiment of this application. The method can be executed by a network attack detection device based on deep reinforcement learning and topology reconstruction. The method includes steps 101-104: Step 101: Collect network traffic data, node status information and security logs in real time, dynamically model the connection relationship and communication behavior between nodes, and generate a dynamic attack graph that includes node vulnerability status, real-time attack path, attack propagation path and real-time threat indicators.

[0021] In this embodiment, network traffic data refers to data packet information transmitted in the network, including but not limited to source IP address, destination IP address, protocol type (such as TCP / UDP), and port number. Network traffic data can be used to understand network communication behavior. Node status information refers to the status information of various devices or hosts in the network, including operating system version, installed services, open ports, and currently running processes. This information can be used to assess system vulnerabilities and potential security threats. Security logs are log files generated by security devices such as firewalls, intrusion detection systems (IDS), and antivirus software, recording information such as suspicious activities, access attempts, and identified malware. These logs provide important references and judgment criteria for potential attack behaviors.

[0022] In this step, distributed probes capture network layer traffic metadata, host process behavior logs, and security device alarm information. Based on graph theory principles, physical devices are abstracted as nodes, and communication links as edges, constructing an initial network topology. Furthermore, in some examples, graph neural networks and sliding window temporal analysis techniques are integrated to jointly encode node attributes (such as system vulnerability identifiers and service openness) and edge dynamic characteristics (such as communication frequency mutations and protocol anomalies). Combined with a threat intelligence correlation engine, the exploit risk level of nodes, potential lateral movement trajectories of attackers, and verified propagation links are labeled in real time, and multi-dimensional threat indicators (combining vulnerability exploitability, behavioral deviation, and contextual relevance) are calculated.

[0023] For example, in the graph structure construction phase, communication pairs are first analyzed based on network traffic data. Specifically, by analyzing source and destination IP addresses, port numbers, and protocol types, device entities exhibiting continuous communication behavior are abstracted as graph nodes, and communication links that meet preset thresholds (such as the number of connections per unit time and the amount of data packets exchanged) are abstracted as directed edges, forming the basic topology skeleton. Node state information (such as operating system fingerprints, open service lists, and patch versions) is standardized and used as the initial attribute vector of the nodes, while abnormal events in security logs (such as multiple login failures and unconventional port scans) are injected as dynamic behavioral features of the edges. The connections and communication behaviors established in this phase constitute the input basis of the graph neural network, where temporal features such as communication frequency abrupt changes and protocol deviations from the baseline are extracted through a sliding window mechanism to form a graph sequence under continuous time slices.

[0024] Subsequently, in the semantic enhancement stage, for example, a spatiotemporal graph neural network architecture is employed to achieve dynamic modeling. The main body of the spatiotemporal graph neural network model includes graph convolutional layers for aggregating structured information of neighboring nodes, gated recurrent unit layers for processing the temporal evolution features of nodes and edges within a sliding window, and a self-attention mechanism for dynamically weighting critical communication paths and high-risk nodes. The model parameters of the spatiotemporal graph neural network in this embodiment include, but are not limited to, hyperparameters such as the receptive field of the graph convolutional kernel, the hidden state dimension of the recurrent unit, the number of attention heads, and the learning rate. Further, optionally, through offline training and online fine-tuning for continuous optimization, the spatiotemporal graph neural network can capture the discrimination boundary between normal communication patterns and abnormal behaviors. This architecture transforms the original topology into a dynamic representation rich in security semantics, providing high-order feature support for attack graph generation.

[0025] It is worth noting that, optionally, the working principle of the threat intelligence correlation engine in the above steps is as follows: First, it connects to multi-source intelligence input, including public vulnerability databases, malicious address lists, attack tactic knowledge bases, and internal historical alert databases. The raw intelligence is standardized, cleaned, and structured, and a composite index system is constructed. For IP address-based intelligence, a prefix tree structure is used to support fast range matching; for hash or domain name-based intelligence, a Bloom filter is deployed to achieve constant-time initial screening, reducing invalid query overhead. In the real-time correlation stage, a streaming computing architecture can be further adopted to compare the traffic metadata and node status collected by distributed probes with the indexed intelligence database in parallel. A sliding window mechanism is used to dynamically analyze communication behavior sequences. Through the lightweight rule engine and similarity matching algorithm deployed within the threat intelligence correlation engine itself, abnormal activities highly consistent with known attack patterns are identified. The threat intelligence correlation engine has a built-in intelligent caching strategy, implementing memory residency for high-frequency query results (such as active malicious IP resolution records) and setting a dynamic expiration mechanism based on intelligence timeliness to balance response speed and resource consumption.

[0026] Furthermore, in the construction of the dynamic attack graph, the correlation results are injected precisely in an incremental manner. When the operating system version of a node matches high-risk vulnerability intelligence, its vulnerability status label and risk weight are updated in real time. When a communication link triggers a malicious behavior pattern, it is automatically marked as a real-time attack path candidate, and the threat indicators of the edges are dynamically adjusted based on the intelligence confidence level. At the same time, the threat intelligence correlation engine is deeply coupled with the graph neural network module, using high-confidence correlation results as prior knowledge input to the model, guiding the attention mechanism to focus on key threat areas, and reducing noise interference in model inference.

[0027] It is evident that the threat intelligence association engine relies on a distributed computing framework to achieve horizontal scaling, supports high-concurrency event stream processing, maintains high throughput while ensuring sub-second latency, and ensures that the construction of dynamic attack graphs has both intelligence-driven accuracy and meets the real-time requirements of highly dynamic network environments.

[0028] In this embodiment, the dynamic attack graph can be implemented as a directed weighted semantic graph that evolves continuously over time. Node vulnerability status refers to the known security flaws associated with a node and their exploitable risk level. For example, a database server with an unpatched remote code execution vulnerability and a high exposure surface is used to quantify the initial intrusion probability. Real-time attack paths refer to active attack chains confirmed by behavioral analysis within the current time window. For example, the communication trajectory of an external IP attempting to access an internal file server via an abnormal SMB connection is used to identify ongoing attack behaviors. Attack propagation paths refer to potential diffusion routes deduced based on graph connectivity and node vulnerabilities. For example, a high-probability path for an infected workstation to move laterally to other hosts on the same network segment is used to guide proactive isolation. Real-time threat indicators are quantitative risk values ​​calculated by integrating vulnerability severity, behavioral deviation, and contextual relevance. For example, a dynamic score generated for a node based on its vulnerability score, abnormal process activity frequency, and traffic leakage characteristics is used for decision-making priority ranking.

[0029] The above parameters are implemented through end-to-end inference of graph neural networks. Specifically, the node embeddings output by the graph neural model are mapped to vulnerability risk levels and threat indicators by a classification head, and the edge embeddings are determined by a binary classification head to determine whether they constitute a real-time attack path. The attack propagation path is derived by combining graph traversal algorithms and embedding similarity calculations. Finally, all security semantic parameters are embedded into the basic topology to form a dynamic attack graph with a complete structure and rich semantics.

[0030] Therefore, dynamic attack graphs not only intuitively present the current network security situation, but also predict attack spread trends through structural evolution analysis of continuous time slices. They transform discrete and heterogeneous security data into structured knowledge graphs with spatiotemporal correlation, improve the interpretability of complex attack chains and the timeliness of threat perception, and further accelerate the detection speed of attack behaviors and attack patterns.

[0031] Step 102: Input the dynamic attack graph into the deep reinforcement learning agent, and obtain the optimal defense action by analyzing the current network state through the deep reinforcement learning agent.

[0032] Understandably, a deep reinforcement learning agent is a core module for autonomous decision-making that integrates deep neural networks and reinforcement learning theory. Essentially, it's an agent system that learns optimal policies through continuous interaction with the network environment. In practical applications, an exemplary structure for a deep reinforcement learning agent is a two-stream architecture consisting of a built-in policy network and a value network. The policy network maps the environment state to an action probability distribution, while the value network evaluates the long-term cumulative reward of the current state.

[0033] In this embodiment, the deep reinforcement learning agent receives a dynamic attack graph state vector transformed by a graph embedding layer. It aggregates the topological features of node neighborhoods through a graph convolutional layer and dynamically focuses on high-risk nodes and key propagation paths using a multi-head self-attention mechanism, forming a deep understanding of the network threat landscape. Offline, the deep reinforcement learning agent performs policy pre-training in a simulation environment built on the ATT&CK framework, continuously optimizing network parameters through a policy gradient algorithm. During online runtime, it performs real-time inference based on a preset reward function. This function comprehensively considers multiple dimensions such as attack blocking effectiveness, service interruption duration, and resource consumption, guiding the deep reinforcement learning agent to achieve an optimal balance between security protection and business continuity. In practical applications, the function metrics can be dynamically adjusted according to network requirements and actual equipment operation.

[0034] The optimal defensive action refers to the sequence of defensive operations calculated by a deep reinforcement learning agent under the current network threat landscape, which maximizes the combined value of long-term security benefits and system availability. This action sequence is not an isolated instruction, but a combination of coordinated strategies generated after weighing multiple objectives, possessing environmental adaptability and execution feasibility.

[0035] For example, the optimal defense action includes, but is not limited to, blocking malicious IPs, isolating infected nodes, redirecting malicious traffic, or triggering topology reconstruction.

[0036] Specifically, blocking malicious IPs refers to prohibiting all network communication from a specific source address through firewall rules. For example, when an external IP continuously initiates abnormal port scans, its access permissions are immediately blocked to quickly cut off external attack entry points and prevent the initial intrusion from escalating. Isolating infected nodes refers to moving hosts with abnormal behavior into isolation zones. For example, after a workstation detects a lateral movement attempt, the SDN controller restricts its communication permissions to interact only with the security analysis platform, curbing the spread of attacks within the internal network and preserving a time window for emergency response. Redirecting malicious traffic refers to directing suspicious data flows to a dedicated analysis environment. For example, internal requests to access abnormal domains are redirected to a honeypot system. This serves to capture attack details without interrupting normal user services, assisting in in-depth analysis and consuming attacker resources. Triggering topology reconfiguration refers to dynamically adjusting the network's logical structure. For example, when ransomware propagation signs are detected, a micro-isolated network segment containing core business servers is automatically created and critical traffic is rerouted, fundamentally changing the attack surface to block potential propagation paths and improve network resilience.

[0037] For example, after inputting a dynamic attack graph, the graph neural network extracts semantic features such as node vulnerability states and attack paths; the policy network calculates the execution probability of each candidate action based on the current state; and the value network simultaneously evaluates the long-term impact of the action sequence. During inference, the agent not only considers the immediate threat mitigation effect but also predicts the subsequent impact of actions on business flows and network connectivity, ultimately outputting a confidence-weighted optimal defense action sequence. Thus, without relying on a manual rule base, the agent can autonomously adapt to unknown attack patterns. Even when facing zero-day exploits or advanced persistent threats, it can still generate accurate and low-perturbation defense strategies, providing timely and robust intelligent decision support for highly dynamic network environments.

[0038] In this step, the dynamic attack graph is transformed into a state vector via a graph embedding layer and input into the deep reinforcement learning agent. The agent employs a policy gradient optimization framework, whose neural network includes graph convolutional modules to extract topological correlation features, combined with an attention mechanism to focus on high-risk areas. The agent learns policies through continuous interaction with the network environment in a simulation setting. Based on a pre-defined reward function (positive incentives for successfully blocking attack propagation, and negative penalties for misoperations affecting critical business operations), it autonomously optimizes its action selection strategy. In actual operation, based on the current attack graph state, the agent comprehensively evaluates the long-term security benefits and business impacts of candidate actions such as isolating nodes, adjusting access control, and traffic scheduling, outputting the optimal defense action sequence that balances immediate blocking effectiveness and system availability. Without relying on traditional predefined rule bases, the end-to-end deep reinforcement learning agent autonomously adapts to various attack patterns, improving the decision-making robustness and generalization ability against zero-day threats and advanced persistent attacks.

[0039] As an optional embodiment, in step 102, firstly, a hierarchical decision-making mechanism is used to actively learn the dynamic attack graph, identify the current network state, and generate collaborative defense actions. The deep reinforcement learning agent includes multiple expert agents and one decision-maker agent. Each expert agent identifies specific attack features contained in the dynamic attack graph and outputs the confidence level corresponding to each identification result. The decision-maker agent generates collaborative defense actions based on the identification results and confidence levels output by each expert agent. Furthermore, the deep reinforcement learning agent continuously interacts with the network environment, optimizing the defense strategy with the goal of maximizing the collaborative defense actions corresponding to the accumulated rewards. Specifically, the deep reinforcement learning agent updates the action value function through temporal difference learning and balances exploration and utilization based on an ε-greedy strategy, gradually converging to the optimal defense action sequence.

[0040] In an optional embodiment of step 102, the deep reinforcement learning agent employs a hierarchical decision-making architecture to realize the parsing of network states and the generation of collaborative defense strategies. This architecture consists of a hierarchical decision-making system composed of multiple expert agents and a decision-maker agent.

[0041] First, a state space integrating dynamic attack graph features is constructed. This space integrates attack path features (covering the evolution sequence of attack phases and the correlation between vulnerability exploitation), real-time traffic anomaly indicators (such as session anomaly rate and protocol deviation per unit time), and node security trustworthiness scores (calculated based on historical behavior and vulnerability exposure surface), providing a structured environmental observation basis for intelligent agents.

[0042] Taking a subnet containing an infected workstation, a database server, and a web application server as an example, the dynamic attack graph is first analyzed. An abnormally high-frequency connection sequence based on the SMB protocol between the workstation and the database server is identified and marked as the lateral movement phase. Structured Query Language (SCL) injection characteristic traffic exists between the database server and the web server, marked as the privilege escalation phase. The real-time traffic anomaly index is calculated using a sliding window, with the formula (number of abnormal sessions / total number of sessions within the window) × 100; the current window's calculated value is 85. The node security trust score uses a weighted fusion model. The workstation trust score is calculated as: 1 - (0.6 × vulnerability severity normalized value + 0.4 × behavior anomaly). The vulnerability severity normalized value is calculated as follows: CVSS score 9.0 / 10, behavior anomaly metric normalized to 0.8, resulting in a workstation trust score of 0.1. The database server and web server are calculated using the same logic. The above attack path sequence characteristics, traffic anomaly index, and node trust score are normalized and concatenated into a 512-dimensional state vector, which is then input into the agent decision module.

[0043] Secondly, a multi-objective weighted reward function is designed to scientifically quantify the defense effect. This function comprehensively evaluates the threat containment effect (such as the integrity of attack path interruption), the impact of network resilience (such as the degree of maintenance of core business service availability), and the cost of defense actions (such as the scope of business interruption caused by isolation operations). Through dynamic weight allocation, the intelligent agent is guided to achieve the optimal trade-off between security protection and business continuity.

[0044] For example, the calculation process of the multi-objective weighted reward function is as follows: The threat containment effect index is (number of successfully cut attack path edges / total number of initial attack path edges) × 100. In the example, 2 edges are cut and the total number of edges is 3, resulting in 66.7. The network resilience impact index is (number of core business nodes that maintain service availability / total number of core nodes) × 100. In the example, all 3 core nodes are normal, resulting in 100. The defense action cost index is (number of normally isolated nodes / total number of network nodes) × (-50). In the example, 1 non-critical node is mistakenly isolated and the total number of nodes is 10, resulting in -5. The reward function R is: first weight parameter × threat containment effect + second weight parameter × network resilience impact + third weight parameter × defense action cost, where the first weight parameter is 0.5, the second weight parameter is 0.3, and the third weight parameter is 0.2, resulting in 62.35. Dynamic weight adjustment mechanism. If three consecutive decisions lead to core business interruption, the second weight parameter automatically increases by 0.1, and the third weight parameter decreases by 0.1 accordingly, continuously optimizing the business continuity assurance capability.

[0045] During the hierarchical decision-making and execution phase, expert agents process specific dimensions of the dynamic attack graph in parallel. Some expert agents focus on identifying lateral movement patterns, others on extracting exploit chain features, and still others analyze traffic temporal anomalies. Each expert agent outputs its identification conclusion and corresponding confidence score. The decision-maker agent receives all expert outputs and, based on confidence scores, weights and fuses multi-source analysis results to generate a sequence of collaborative defense actions, including blocking, isolation, redirection, and topology adjustments. To enhance decision reliability, a confidence-based active learning mechanism is embedded. When the decision-maker agent's confidence in the output action falls below a preset threshold, a simulated expert feedback loop is automatically triggered, injecting high-confidence historical cases and synthetic adversarial examples into the training process. This allows for targeted fine-tuning of the policy network, improving the agent's discrimination capabilities in ambiguous threat scenarios.

[0046] For example, the lateral movement recognition expert agent outputs a confidence score of 0.92, the exploit chain expert agent outputs a confidence score of 0.78, and the traffic anomaly expert agent outputs a confidence score of 0.85. The decision-maker agent allocates fusion weights based on historical accuracy (lateral movement expert 0.4, exploit chain expert 0.3, traffic anomaly expert 0.3), calculates the weighted confidence score = 0.4 × 0.92 + 0.3 × 0.78 + 0.3 × 0.85 = 0.833, and generates a collaborative action to isolate the infected workstation and block its traffic to the database server. When the decision confidence score is lower than the threshold of 0.8, an active learning loop is triggered. Further, taking the retrieval of 5 high-confidence historical samples as an example, 3 perturbation samples are synthesized using a generative adversarial network, and the policy network is fine-tuned for 3 rounds with a learning rate of 0.001. Policy optimization uses temporal difference learning, and the action value function update formula is expressed as: ,in, 0.1 It is 0.9. Specifically, This represents the updated action value estimate, which is the optimized evaluation result of the long-term benefit of the agent after performing a specific action in the current state. This represents the original action value estimate stored before the update, corresponding to the existing knowledge of the state-action pair. The learning rate, ranging from 0 to 1, controls the strength of the correction of model parameters by new experience. A typical value is set to 0.1 to balance convergence stability and learning efficiency. The scalar reward is a real-time feedback from the environment, dynamically generated by a preset reward function based on the effect of the defensive action. This is a discount factor, ranging from 0 to 1, with a typical value of 0.9. It is used to quantify the current value weight of future accumulated rewards. The closer the value is to 1, the more emphasis is placed on long-term safe returns. This indicates the next observation state the network environment transitions to after a defensive action is performed. Indicates the state The set of all available defensive actions. Indicates the state Next, traverse all candidate actions. The maximum action value estimate obtained later reflects the agent's forward-looking prediction of the optimal strategy for the future. The entire value within the parentheses in the formula constitutes the temporal difference error, i.e., the deviation between the target value and the current estimate, which drives... The value is iteratively optimized along the error gradient direction. This update mechanism allows the agent to continuously calibrate its strategy through interactive experience without requiring a complete environmental model, effectively supporting the adaptive evolution of network defense decisions in dynamic threat environments.

[0047] At the strategy optimization level, the agent continuously interacts with the network environment through a temporal difference learning algorithm, iteratively updating the action value function parameters, and employs an ε-greedy strategy to dynamically balance the ratio of exploring new defense strategies with utilizing verified effective actions, ensuring that the strategy steadily converges to the global optimum during training. For example, simultaneously using an ε-greedy strategy, the initial exploration probability ε is set to 0.2, decaying by 0.01 every 100 decisions, dynamically balancing exploration and utilization to ensure that the strategy steadily converges to the global optimum.

[0048] This layered architecture not only enables refined analysis of the characteristics of multi-source heterogeneous threats, but also, through confidence-driven active learning and multi-objective reward guidance, makes defense decisions highly accurate, adaptable, and business-friendly, effectively addressing the complex challenges posed by advanced persistent threats and zero-day attacks.

[0049] Optionally, prior to step 102, a state space for the deep reinforcement learning agent can be constructed. This state space is composed of attack path features extracted from the dynamic attack graph, real-time traffic anomaly index features, and security trustworthiness scores for each node. The attack path features include attack phase sequences, vulnerability exploitation relationships, and lateral movement paths. Furthermore, an action space for the deep reinforcement learning agent is defined, comprising a set of defensive actions: blocking malicious IP addresses, restricting access to specific ports, isolating infected network devices, redirecting malicious traffic to the honeypot system, and triggering network topology reconfiguration commands. Additionally, a reward function for the deep reinforcement learning agent is set. This function dynamically calculates reward values ​​based on network state changes after the execution of defensive actions. The reward value calculation comprehensively considers threat blocking effectiveness, service availability impact, and defense resource overhead. Positive rewards correspond to successful attack containment and improved network resilience, while negative rewards correspond to false alarms and service interruptions.

[0050] Specifically, when constructing the state space of a deep reinforcement learning agent, a ransomware attack on a financial data center is used as an example. The attack path features extracted from the dynamic attack graph include the attack phase sequence (initial intrusion phase exploiting a remote code execution vulnerability, lateral movement phase spreading through the internal SMB protocol, and data encryption phase locking the file server), vulnerability exploitation relationships (clearly indicating the presence of the high-risk CVE-2023-1234 vulnerability on infected nodes), and lateral movement paths (clearly showing the communication link from the terminal device to the core database server). Real-time traffic anomaly indicators are calculated using a sliding window statistical method; for example, an infected terminal makes 50 abnormal connection requests per second, exceeding the historical baseline by 30. The security trustworthiness score for each node is calculated using a weighted model, with the formula: Trustworthiness = 1 - (Vulnerability Weight × Vulnerability Severity Normalized Value + Behavior Weight × Behavior Anomaly). The vulnerability weight is set to 0.6, the behavior weight to 0.4, and the file server, due to an unpatched high-risk vulnerability and abnormal traffic, receives a trustworthiness score of 0.15. The aforementioned multidimensional features are normalized and fused into a 512-dimensional state vector, enabling the agent to accurately capture the spatiotemporal evolution of threats and improve the perception granularity and situational understanding depth of complex attack chains.

[0051] The action space explicitly defines a set of defensive actions consisting of five categories. Blocking malicious IP addresses refers to prohibiting communication from specific source addresses in firewall rules. Restricting access to specific ports refers to closing unnecessary service ports to reduce the attack surface. Isolating infected network devices refers to logically moving abnormal nodes into an isolation zone. Redirecting malicious traffic to a honeypot system refers to guiding suspicious traffic to a spoofed environment. Triggering network topology reconfiguration commands refers to dynamically adjusting logical connections through a software-defined network controller. A honeypot system is an active decoy technology that attracts attackers to interact and probe by deploying highly simulated fake services or system environments. It captures attack methods, delays attack processes, and generates high-value threat intelligence in an isolated environment, while effectively protecting real business assets from direct harm. This action set design ensures that agents can generate multi-layered collaborative strategies, balancing immediate blocking with defense-in-depth requirements.

[0052] For example, in a positive reward scenario, when attack traffic completely disappears after the infected node is isolated, and core business services continue to operate normally, it indicates that the threat has been effectively contained and network resilience has been enhanced. The system awards a positive reward to reinforce this decision-making pattern. Conversely, in a negative reward scenario, when a defensive action mistakenly identifies normal business traffic as malicious and blocks it, causing a temporary interruption of critical services, or when isolation operations affect healthy nodes and cause business access failures, it indicates a decision-making bias. A negative reward is awarded to suppress such erroneous operations. Real-time feedback guides the agent to continuously optimize strategies, ensuring security effectiveness while strictly maintaining business continuity.

[0053] Thus, the fusion of multi-dimensional features in the state space enables the agent to possess global situational awareness, the structured definition of the action space ensures the executability and diversity of policy generation, and the multi-objective reward function drives policy optimization to always revolve around the dual objectives of security benefits and business costs. The introduction of honeypot systems enhances threat intelligence acquisition capabilities, the collaborative design of defense action sets improves response flexibility, and the refined positive and negative reward mechanism effectively suppresses the risks of over-defense and misoperation. This allows the agent to generate highly efficient and low-disturbance defense strategies, improving the network system's proactive immunity, dynamic adaptability, resilience, and business friendliness against advanced persistent threats and zero-day attacks, providing a verifiable and self-evolving intelligent security protection foundation for highly dynamic scenarios such as cloud computing and the Internet of Things.

[0054] As an optional embodiment, in step 102, after inputting the dynamic attack graph into the deep reinforcement learning agent and obtaining the optimal defense action by analyzing the current network state through the deep reinforcement learning agent, the method further includes: using the network topology reconstruction result and the dynamic execution result of the optimal defense action as reward feedback to dynamically update the policy network of the deep reinforcement learning agent, thereby achieving continuous threat protection in a highly dynamic network environment.

[0055] For example, taking the detection of a ransomware lateral movement attack in a cloud data center as an example, the deep reinforcement learning agent outputs defense instructions to isolate infected virtual machines and trigger micro-segment topology reconstruction. The policy execution unit calls the SDN controller API to complete logical isolation and business traffic rerouting. Real-time execution feedback data is collected, such as a 92% decrease in attack traffic, 100% availability of core database services, but a non-critical test node being mistakenly isolated, causing a 15-second service interruption. The reward calculation module generates feedback signals based on a multi-objective function: the threat blocking effect index is 92, the service availability impact index is 100, and the defense resource overhead index is -1.5. The base reward R_base equals 0.5 multiplied by 92 plus 0.3 multiplied by 100 plus 0.2 multiplied by -1.5, resulting in 75.7. Further, event-level rewards are added: a reward of +2 for successfully cutting off the attack path, a reward of +3 for meeting service availability targets, and a penalty of -3 for mistaken isolation, resulting in a total reward R_total of 77.7. The reward value, along with the reconstructed dynamic attack graph (showing broken attack paths and increased node credibility), constitutes the training sample and is input into the policy network. A near-end policy optimization algorithm is used to calculate the policy gradient, updating network parameters with a learning rate of 0.001 to strengthen the decision-making tendency to accurately isolate high-risk nodes and protect the core business topology. Through multiple consecutive interactive iterations, the agent reduces the false isolation rate and improves the timeliness of attack blocking in subsequent decisions.

[0056] Step 103: Generate policy instructions based on the optimal defense action and send them to the network policy execution point.

[0057] The policy enforcement points include firewalls, intrusion prevention systems, and software-defined network controllers. A policy enforcement point is a set of automated components deployed in critical network locations that can receive standardized instructions from the policy orchestration engine and execute specific security operations.

[0058] Understandably, firewalls, as the fundamental protection unit at the network boundary, implement fine-grained access control on incoming and outgoing traffic based on preset security policies, effectively isolating unauthorized access and supporting micro-segmentation technology for internal traffic management. Intrusion prevention systems, through packet reassembly, deep parsing of application-layer protocols, and real-time matching of signature databases, identify and proactively block application-layer threats such as buffer overflows, cross-site scripting, and injection attacks within milliseconds, providing bidirectional protection for both internal and external networks. Software-defined network controllers, acting as the intelligent scheduling hub of the network architecture, rely on a control plane and data plane separation architecture. Through southbound interfaces, they dynamically adjust switch flow tables and logical topologies to achieve second-level isolation of infected nodes, intelligent rerouting of critical business traffic, and on-demand construction of isolated micro-segments. Seamless integration of standardized APIs and a policy orchestration engine forms a defense-in-depth system from traffic filtering and deep detection to topology reconstruction, upgrading traditional static protection into a dynamic security closed loop with environmental awareness, policy adaptation, and structural self-healing capabilities. This improves response speed and containment effectiveness against complex attacks such as lateral movement and ransomware propagation in highly dynamic network scenarios.

[0059] In this step, the abstract defense actions output by the agent are parsed into a standardized set of instructions recognizable by the policy orchestration engine. Based on a pre-defined device capability model, semantic actions such as isolating infected nodes are mapped to firewall access control list update instructions, intrusion prevention system deep detection policy activation instructions, and software-defined network controller flow table modification instructions, etc. All instructions, after being digitally signed and verified for integrity, are distributed to the corresponding execution points according to priority through a secure communication channel. The policy orchestration engine supports instruction conflict detection and execution order optimization, ensuring that high-risk actions are implemented first. This step achieves a seamless transition from defense decisions to physical operations, effectively eliminating response delays and operational errors caused by manual intervention, while ensuring compatibility with heterogeneous security devices from multiple vendors through a unified interface specification.

[0060] As an optional embodiment, in step 103, the optimal defense action is encoded into an executable policy instruction set. In this embodiment, the policy instructions in the policy instruction set at least include an action type, a target object identifier, execution parameters, and a priority flag. The target object identifier includes a malicious IP address, an infected node ID, or an abnormal traffic session ID, and the execution parameters include blocking duration, isolation range, redirection target honeypot address, or edge-biconnected component algorithm parameters for topology reconstruction.

[0061] Further optionally, in the policy instruction set, blocking malicious IP actions correspond to the firewall's dynamic access control instructions, isolating infected nodes actions correspond to the intrusion prevention system's process isolation instructions, and triggering topology reconstruction actions correspond to the software-defined network controller's flow table update instructions.

[0062] For example, during the policy instruction coding phase, the optimal defense action is structured and converted into a standardized policy instruction set. Each instruction includes an action type (such as BLOCK_IP, ISOLATE_NODE), a target object identifier (malicious IP address 192.168.1.100, infected node ID NODE_007, abnormal session ID SESSION_20240520_001), execution parameters, and a priority flag (values ​​from 1 to 5, with 1 being the highest). Execution parameters are dynamically set based on the action. Specifically, the blocking duration is set to 3600 seconds (for high-risk attack scenarios), the isolation range is specified as the subnet 10.0.0.0 / 24 to which the node belongs, the redirection target honeypot address is set to 10.10.10.100, and the topology reconstruction parameters use an edge-biconnected component algorithm with a minimum connectivity threshold of 2. The target object identifier is directly extracted from the node / edge attributes of the dynamic attack graph. The execution parameters are generated by the agent based on the threat level (blocking duration is set to 3600 seconds when CVSS score ≥ 8.0) and business impact assessment. Priority flags are determined by the policy orchestration engine based on a preset rule base (blocking actions are fixed at 1, and isolation actions are fixed at 2).

[0063] For example, policy directives for firewalls configure access control rules via a RESTful API, policy directives for intrusion prevention systems update the detection signature database via the SNMP protocol, and policy directives for software-defined network controllers (SDNs) dynamically modify flow table entries via the OpenFlow protocol to achieve real-time blocking or redirection of malicious traffic. For instance, firewall directives configure access control rules via a RESTful API (endpoint https: / / aaa-api.example.com / v1 / rules, timeout 5 seconds). Intrusion prevention system directives update detection policies via the SNMP protocol (community name public, 3 retries). SDN controller directives construct FLOW_MOD messages following the OpenFlow 1.3 protocol (matching fields include source IP and destination port, action field is set to DROP or OUTPUT to honeypot port). Protocol parameters are pre-defined by the device capability model, which automatically loads the API specifications and parameter constraints supported by each execution point during initialization via device fingerprinting.

[0064] Furthermore, for multiple policy instructions in the policy instruction set, these instructions are sorted by priority, and the sorting results are subjected to timing logic conflict detection to verify the consistency between the policy instructions and avoid service interruptions caused by policy conflicts. For example, for policy instructions for topology reconstruction, the SDN controller first calculates redundant paths and pre-configures flow tables, and then triggers link switching operations to ensure the continuity of network services during the reconstruction process.

[0065] For example, the commands are first sorted in ascending order by priority. If priorities are the same, a pre-defined sequence is used based on the action type (blocking takes precedence over isolation). Then, a timing logic conflict detection is performed, an instruction dependency graph is constructed, and operation similarity is calculated (threshold set to 0.8). If mutually exclusive instructions are detected (e.g., rules allowing and blocking the same IP address exist simultaneously), the parameters of lower-priority instructions are automatically adjusted or their execution is delayed. Topology reconfiguration instructions use a pre-configuration mechanism. The SDN controller first calls Dijkstra's algorithm to calculate redundant paths (maximum hop count limited to 5), pre-sets backup flow table entries (idle timeout of 300 seconds), monitors link switching jitter (threshold of 100 milliseconds), and automatically rolls back when limits are exceeded to ensure zero-interruption service switching.

[0066] Finally, after successful verification, the application programming interfaces of the firewall, intrusion prevention system, and software-defined network controller are invoked to send multiple policy instructions to the corresponding network policy execution points.

[0067] For example, after successful verification, the application programming interfaces (APIs) of each policy enforcement point are called sequentially according to priority markers. For the firewall, a POST request is sent to the endpoint via HTTPS. The request body includes the action type "deny", source IP address 192.168.1.100, destination IP address 10.0.0.5, protocol type TCP, destination port 445, and blocking duration 3600 seconds. The parameter values ​​are obtained from the policy directive parsing, where the IP address and port are taken from the node and edge attribute fields of the dynamic attack graph, and the blocking duration is dynamically calculated based on a CVSS score of 9.0 using the formula 3600 multiplied by the score and divided by 10. The firewall returns the rule identifier RULE_20240520_001 and the effective timestamp 2024-05-20T14:30:00Z. For the intrusion prevention system, a SET request is sent to the device IP address 10.0.0.10 via SNMPv3 protocol, with OID 1.3.6.1.4.1.9999.1.1.1, setting the redirection target IP address to 10.10.10.100. The matching conditions are source IP address 192.168.1.100 and destination port 445. Parameters are extracted from the command set by the policy orchestration engine, and an SNMP response status code of 0 indicates successful configuration. For the software-defined network controller, the northbound REST API endpoint is called to send a flow table add command, specifying the switch DPID 00:00:00:00:00:00:00:01. The matching fields include source IP address 192.168.1.100, destination IP address 10.0.0.5, IP protocol number 6, and TCP destination port 445. The action is to output to the honeypot port. The DPID is obtained by querying the network topology mapping table (initialized and constructed by the LLDP protocol), and the controller returns the flow table entry identifier FLOW_20240520_001. The status codes, rule identifiers, and timestamps returned by all execution points are recorded in real time to the feedback acquisition module for subsequent verification of the effective status of instructions (such as rule hit rate statistics and isolation node connection number monitoring), and serve as input for updating the environment state to drive the deep reinforcement learning agent's policy iterative optimization, forming a closed-loop adaptive defense system.

[0068] Optionally, feedback data from policy execution points can be collected in real time, including command execution status, network traffic changes, and node availability metrics, and used as input for environmental state updates to the deep reinforcement learning agent. For example, policy execution points may return status codes (200 for success) and execution time. Distributed probes may report traffic change metrics (blocked packet rate, business traffic volatility). Node monitoring agents may collect availability data (service response time, CPU utilization). After normalization, the feedback data is fused with the updated dynamic attack graph to generate a new state vector, which is then input into the deep reinforcement learning agent. For example, if the core business traffic volatility is below 5% and threat traffic returns to zero after a blocking command is executed, a positive reward signal is generated. If the isolation operation causes the critical service response time to exceed a threshold (e.g., 200 milliseconds), a negative reward is triggered. The feedback collection interval is set to 1 second, and the data window is retained for 300 seconds to ensure the agent receives timely environmental feedback.

[0069] Step 104: By calling the application programming interface of the policy execution point, the system dynamically executes actions such as blocking malicious IPs, isolating infected nodes, or redirecting malicious traffic, and triggers network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving real-time detection and interception of attack behaviors.

[0070] In this step, each policy execution point implements commands instantly via a standardized application programming interface (API). Firewall rules are updated to block communication from malicious source addresses, the intrusion prevention system performs deep cleaning of specified traffic, and the software-defined network controller dynamically adjusts flow tables to redirect suspicious traffic to the analysis sandbox. A network topology reconfiguration mechanism is triggered simultaneously. Based on attack path analysis, logical connections between high-risk nodes and core business domains are disconnected, and isolated micro-segments or redundant transmission paths are created for critical services, achieving software-defined level dynamic adjustment of physical connection relationships. The reconfiguration process employs a smooth switching strategy to avoid service disruptions and continuously collects execution feedback (such as traffic changes and node status) and transmits it back to the dynamic modeling unit, driving attack graph updates and iterative optimization of agent policies. This step not only enables rapid interception of attacks but also effectively improves the network's resilience after being impacted through topology elastic reconfiguration.

[0071] As an optional embodiment, in step 104, dynamically executing actions such as blocking malicious IPs, isolating infected nodes, or redirecting malicious traffic by calling the application programming interface of the policy execution point includes: First, based on the type of the optimal defense action and the target object identifier, an executable instruction sequence matching the target network policy enforcement point is generated. Specifically, blocking malicious IP actions are converted into dynamic access control list rules for the firewall, with the rule content including the source IP address, destination IP address, port number, protocol type, and blocking duration. Isolating infected nodes actions are converted into process isolation instructions for endpoint security agents or port disabling commands for software-defined network controllers. Redirecting malicious traffic actions are converted into traffic redirection policies for intrusion prevention systems, redirecting suspicious sessions to the honeypot system for in-depth behavioral analysis.

[0072] For example, the source IP address refers to the IP of the terminal or server initiating the malicious attack, extracted from the real-time attack path characteristics of the dynamic attack graph and determined by combining the abnormal source IP annotations in the security log, such as 192.168.1.105. The destination IP address is the IP of the core business node of the attack target, such as 10.0.0.20, obtained from the endpoint node attributes of the attack propagation path in the dynamic attack graph. The port number is the high-risk port used by the attack, such as 445, obtained by filtering the port with the highest abnormal access frequency from the protocol port statistics of network traffic data. The protocol type is determined based on the protocol field parsing of the traffic data packet and matching with the attack feature database, such as the TCP protocol. The blocking time is dynamically calculated based on the threat level: 3600 seconds for a CVSS score of 9.0 and above, 1800 seconds for 6.0 to 8.9, and 300 seconds for below 6.0, to avoid permanent blocking leading to erroneous operations. The action of isolating infected nodes is converted into a process isolation command from the endpoint security agent or a port disabling command from the software-defined network controller. The process isolation command from the endpoint security agent must include the node ID, isolation scope, and target process ID. The node ID is extracted from the node attributes of the dynamic attack graph and associated with the endpoint security agent's device registration list, such as NODE-PC-072. The isolation scope is preset to only allow communication with the security management platform, such as 172.16.0.50. The target process ID is determined from the abnormal process list in the node status information combined with malicious process alerts in the security logs, such as PID1234. The port disabling command from the software-defined network controller must include the switch DPID and physical port number. The switch DPID is automatically discovered and registered from the SDN controller's device management list via the LLDP protocol, such as 00:00:00:00:00:00:00:08. The physical port number is determined from the node-link association characteristics of the dynamic attack graph combined with the MAC address binding table of the switch port, such as Gi1 / 0 / 24. Redirecting malicious traffic actions into traffic redirection strategies for intrusion prevention systems requires the inclusion of an abnormal session ID, honeypot address, and redirection priority. The abnormal session ID is obtained by filtering sessions that match the attack characteristics from the session tracking module of network traffic data, such as SESS-20240601-0089. The honeypot address is randomly selected from a preset honeypot cluster address pool of available nodes, such as 10.10.10.200. The redirection priority is fixed at 2, which is higher than normal forwarding but lower than core business traffic.

[0073] Furthermore, the policy coordination engine performs atomicity verification and conflict resolution on multiple defense actions executed in parallel within the executable instruction sequence. When resource contention or logical conflicts are detected between policy instructions, the execution order is dynamically adjusted based on temporal logic and network topology, prioritizing the isolation of critical nodes along the attack path, triggering traffic redirection and IP blocking actions, and ensuring the consistency and efficiency of the defense policy.

[0074] Specifically, during the atomicity verification phase, the policy coordination engine performs pre-execution simulation on the instruction sequence to check for resource contention, such as the situation where the same switch port is simultaneously disabled and forwarded.

[0075] During the conflict resolution phase, when resource contention or logical conflicts are detected between policy instructions, the execution order is dynamically adjusted based on temporal logic and network topology. The core principles are prioritizing the isolation of critical nodes, blocking the attack source, and minimizing business impact. Isolation operations on critical nodes along the attack path are executed first, followed by traffic redirection and IP blocking. The determination of critical nodes is derived by reverse engineering the node security trust score in the dynamic attack graph; the lower the trust score, the higher the node weight. Weights range from 1 to 10; for example, a database node weight is set to 9, prioritizing its isolation. Simultaneously, an instruction dependency threshold is set (default 0.8) to determine dependencies between instructions. The conflict detection window is set to 5 seconds (3 seconds for WAN scenarios) or 3 seconds for LAN scenarios to avoid misjudgments across time periods and ensure the consistency and efficiency of the defense strategy. For example, when there are two conflicting commands: disabling switch port Gi1 / 0 / 24 and forwarding ERP server traffic through Gi1 / 0 / 24, the forwarding path of ERP server traffic should be adjusted first, and then the port disabling command should be executed. When there are three sets of commands: blocking malicious IPs, isolating infected nodes, and redirecting malicious traffic, they should be executed in the order of blocking malicious IPs, isolating infected nodes, and redirecting malicious traffic, with intervals of 0 milliseconds, 100 milliseconds, and 200 milliseconds respectively.

[0076] Next, the optimized executable instruction sequence is triggered, and network status feedback data is monitored in real time. This network status feedback data includes firewall rule hit rate, isolated node resource status, and redirected traffic session characteristics. For example, isolated node resource status includes the CPU utilization and network connection count of the isolated node. CPU utilization is collected every 5 seconds by the endpoint security agent; a decrease from 80% to 25% indicates effective isolation. The number of network connections is counted using the netstat command; the remaining number of connections after excluding allowed IPs must meet the isolation range requirements, such as retaining only 3 connections to the security platform. Redirected traffic session characteristics include redirected session retention time and the number of redirected traffic packets. Session retention time is counted every 10 seconds by the honeypot system logs; a retention time of 180 seconds indicates successful capture of attack behavior. The number of redirected traffic packets is cumulatively queried using the intrusion prevention system's SNMP protocol to ensure the integrity of traffic redirection.

[0077] Finally, based on the difference between the network status feedback data and the expected defense effect, the failed or partially failed strategies in the executable instruction sequence (such as malicious IP switching or bypassing of isolation nodes) are identified, and the executable instruction sequence corresponding to the optimal defense action is dynamically adjusted, regenerating an executable instruction sequence adapted to the current attack evolution. Simultaneously, an attack evolution coefficient is introduced to quantify the severity of attack behavior changes. This coefficient is obtained by calculating the structural similarity between two consecutive frames of the dynamic attack graph; the lower the similarity, the higher the evolution coefficient, ranging from 0 to 1. For example, an evolution coefficient of 0.7 indicates a drastic attack change. For failed or partially failed strategies, an executable instruction sequence adapted to the current attack evolution is regenerated based on the attack evolution coefficient. For example, after a malicious IP switch, a new malicious IP address is extracted from the dynamic attack graph, and the firewall's dynamic access control list rules are updated. If an isolation node is bypassed, the isolation scope is expanded, and the port disabling command of the software-defined network controller or the process isolation command of the endpoint security agent is adjusted to ensure that the defense strategy always adapts to changes in the attack situation.

[0078] As an optional embodiment, step 104, triggering network topology reconstruction to quickly repair or isolate faulty nodes, includes: when the deep reinforcement learning agent outputs an instruction to trigger topology reconstruction, or when the intrusion detection system confirms a specific threat, analyzing the connectivity of the current network topology, identifying the edge-biconnected components where the attacked node is located, and calculating a set of redundant paths that do not contain the attacked node. The specific threat includes lateral movement attacks or node compromise. Furthermore, based on the calculated set of redundant paths, the network topology is dynamically adjusted by disconnecting malicious connections, enabling redundant links, or migrating critical services to cut off attack paths and restore network functionality.

[0079] Understandably, when the deep reinforcement learning agent outputs a command to trigger topology reconstruction, or when the intrusion detection system confirms a specific serious threat such as a lateral movement attack or node compromise, the topology reconstruction process is automatically initiated. During this process, based on the edge-biconnected component reconstruction algorithm, critical bridging edges and redundant paths in the current network topology are identified. Flow table rules are then issued by the software-defined network controller to dynamically disconnect connections to malicious or compromised nodes. Simultaneously, preset redundant links are enabled or critical services are migrated to secure nodes to ensure the network maintains the reliability of critical service connections during attacks, achieving rapid fault isolation and resilient service recovery.

[0080] Specifically, a reconstruction strategy is dynamically selected based on the severity of the attack and business continuity requirements. If the attacked node is a critical service node, a service migration strategy is used to seamlessly switch the services carried by the attacked node to a pre-calculated redundant node. If the malicious node is in a non-critical isolation zone, a link isolation strategy is used to directly disconnect the connection with the malicious node and activate a backup link to maintain network connectivity.

[0081] The topology adjustment operation is executed in stages by issuing flow table rules through a software-defined network controller. Priority is given to isolating critical nodes along attack paths, enabling redundant links, and verifying network connectivity to ensure uninterrupted critical service traffic during the reconstruction process. Network resilience metrics after reconstruction are monitored in real time, including inter-node path redundancy, attack path isolation efficiency, and service recovery time. These metrics are then fed back to the reward function of the deep reinforcement learning agent, forming a closed-loop optimization mechanism for the topology reconstruction strategy.

[0082] In this embodiment of the application, by constructing a collaborative defense scheme of multi-source data dynamic modeling, deep reinforcement learning intelligent decision-making and software-defined network topology reconstruction, an automated processing flow from attack perception to network self-healing can be realized, achieving proactive defense with environmental adaptability and policy self-adaptation capabilities. This reduces security response latency and the risk of lateral attack spread, bringing users a reliable network security protection experience, and reducing the frequency of manual operation and maintenance intervention and the comprehensive losses caused by business interruption.

[0083] The above describes a network attack detection method based on deep reinforcement learning and topology reconstruction in the embodiments of this application. The following describes the network attack detection device (e.g., a server) based on deep reinforcement learning and topology reconstruction that performs the above network attack detection method.

[0084] See Figure 2 ,like Figure 2 The diagram illustrates a network attack detection device based on deep reinforcement learning and topology reconstruction, which can be applied to servers or terminal devices in network attack detection scenarios requiring defense against network attacks. The network attack detection device based on deep reinforcement learning and topology reconstruction in this embodiment can achieve the corresponding... Figure 1The embodiments described above illustrate the various steps of the network attack detection method based on deep reinforcement learning and topology reconstruction. The functions of the network attack detection device based on deep reinforcement learning and topology reconstruction can be implemented in hardware or by executing corresponding software within hardware. The hardware or software includes one or more modules corresponding to the above functions, and these modules can be software and / or hardware. The network attack detection device based on deep reinforcement learning and topology reconstruction may include a modeling module 601, a decision module 602, and an execution module 603. The functional implementation of each module can be found in [reference needed]. Figure 1 The operations performed in the corresponding embodiments are similar and will not be described again below.

[0085] The modeling module 601 is configured to collect network traffic data, node status information and security logs in real time, dynamically model the connection relationship and communication behavior between nodes, and generate a dynamic attack graph that includes node vulnerability status, real-time attack path, attack propagation path and real-time threat indicators. The decision module 602 is configured to input the dynamic attack graph into a deep reinforcement learning agent, analyze the current network state through the deep reinforcement learning agent, obtain the optimal defense action, generate policy instructions based on the optimal defense action, and send them to the network policy execution point; the policy execution point includes firewalls, intrusion prevention systems, and software-defined network controllers. The execution module 603 is configured to dynamically block malicious IPs, isolate infected nodes, or redirect malicious traffic by calling the application programming interface of the policy execution point, and trigger network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving real-time detection and interception of attack behaviors.

[0086] In this embodiment, the synergistic effect of the above modules enables proactive perception and intervention of attack behaviors, and achieves intelligent decision-making and dynamic self-healing of network topology through deep reinforcement learning.

[0087] This application also provides a terminal device, such as... Figure 3 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example, Figure 3 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 3The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that... Figure 3 The mobile phone structure shown is not intended to limit the scope of the phone and may include more or fewer components, combinations of certain components, or different component arrangements. Although not shown, the phone may also include a camera, Bluetooth module, etc., which will not be described further here.

[0088] In this embodiment of the application, the processor 1080 included in the mobile phone can implement and control other modules to perform the functions performed by the network attack detection device based on deep reinforcement learning and topology reconstruction.

[0089] This application also provides a server; please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 may be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0090] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. The steps performed by the server in the above embodiments can be based on this... Figure 4The structure of server 1100 is shown. For example, the central processing unit 1122 performs the following operations by calling instructions in memory 1132: It collects network traffic data, node status information, and security logs in real time through input / output interface 1158; dynamically models the connection relationships and communication behaviors between nodes; and generates a dynamic attack graph containing node vulnerability status, real-time attack paths, attack propagation paths, and real-time threat indicators. The dynamic attack graph is then input into a deep reinforcement learning agent, which analyzes the current network state to obtain the optimal defense action. Based on the optimal defense action, policy instructions are generated and sent to network policy execution points. These policy execution points include firewalls, intrusion prevention systems, and software-defined network controllers. By calling the application programming interface of the policy execution points, it dynamically executes actions such as blocking malicious IPs, isolating infected nodes, or redirecting malicious traffic, and triggers network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving immediate detection and interception of attack behaviors.

[0091] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0093] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0094] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0096] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0097] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A network attack detection method based on deep reinforcement learning and topology reconstruction, characterized in that, The method includes: Real-time collection of network traffic data, node status information and security logs; dynamic modeling of the connection relationship and communication behavior between nodes; generation of dynamic attack graphs including node vulnerability status, real-time attack paths, attack propagation paths and real-time threat indicators. The dynamic attack graph is input into a deep reinforcement learning agent, which analyzes the current network state to obtain the optimal defense action. Based on the optimal defense action, policy instructions are generated and sent to network policy enforcement points; the policy enforcement points include firewalls, intrusion prevention systems, and software-defined network controllers. By calling the application programming interface of the policy execution point, the system can dynamically block malicious IPs, isolate infected nodes, or redirect malicious traffic, and trigger network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving real-time detection and interception of attack behaviors.

2. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 1, characterized in that, The optimal defense actions include blocking malicious IPs, isolating infected nodes, redirecting malicious traffic, or triggering topology reconstruction.

3. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 2, characterized in that, The step of inputting the dynamic attack graph into a deep reinforcement learning agent, analyzing the current network state through the deep reinforcement learning agent, and obtaining the optimal defense action includes: A hierarchical decision-making mechanism is adopted to actively learn the dynamic attack graph, identify the current network state, and generate collaborative defense actions. The deep reinforcement learning agent includes multiple expert agents and one decision-maker agent. Each expert agent identifies specific attack features contained in the dynamic attack graph and outputs the confidence level corresponding to each identification result. The decision-maker agent generates collaborative defense actions based on the identification results and confidence levels output by each expert agent. By continuously interacting with the network environment through a deep reinforcement learning agent, the defense strategy is optimized with the goal of maximizing the collaborative defense actions corresponding to the accumulated rewards. Specifically, the deep reinforcement learning agent updates the action value function through temporal difference learning and balances exploration and utilization based on an ε-greedy strategy, gradually converging to the optimal defense action sequence.

4. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 3, characterized in that, Before inputting the dynamic attack graph into the deep reinforcement learning agent and analyzing the current network state to obtain the optimal defense action, the process further includes: Construct a state space for a deep reinforcement learning agent, wherein the state space is composed of attack path features extracted from the dynamic attack graph, real-time traffic anomaly index features, and security and trustworthiness scores of each node. The attack path features include attack phase sequence, vulnerability exploitation relationship, and lateral movement path. Define the action space of the deep reinforcement learning agent, which includes a set of defensive actions such as blocking malicious IP addresses, restricting access to specific ports, isolating infected network devices, redirecting malicious traffic to the honeypot system, and triggering network topology reconstruction instructions; A reward function is set for the deep reinforcement learning agent. The reward function dynamically calculates the reward value based on the network state changes after the defense action is executed. The reward value calculation comprehensively considers the threat blocking effect, the impact of service availability, and the defense resource overhead. Positive rewards correspond to successful attack containment and improved network resilience, while negative rewards correspond to false alarms and service interruptions.

5. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 1, characterized in that, The step of generating policy instructions based on the optimal defense action and sending them to the network policy execution point includes: The optimal defense action is encoded into an executable set of strategy instructions; For multiple policy instructions in the policy instruction set, the policy instructions are sorted by priority, and the sorting results are checked for timing logic conflicts to verify the consistency between policy instructions and avoid service interruption due to policy conflicts. After verification, the application programming interfaces of the firewall, intrusion prevention system and software-defined network controller are invoked to send multiple policy instructions to the corresponding network policy execution points. Among them, policy directives for firewalls configure access control rules through RESTful APIs, policy directives for intrusion prevention systems update the detection signature database through SNMP protocol, and policy directives for software-defined network controllers dynamically modify flow table entries through OpenFlow protocol, so as to achieve real-time blocking or redirection of malicious traffic.

6. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 5, characterized in that, The policy instructions in the policy instruction set must include at least the action type, target object identifier, execution parameters, and priority flag; The target object identifier includes a malicious IP address, an infected node ID, or an abnormal traffic session ID, and the execution parameters include blocking duration, isolation range, redirection target honeypot address, or edge biconnected component algorithm parameters for topology reconstruction.

7. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 1, characterized in that, The dynamic execution of blocking malicious IPs, isolating infected nodes, or redirecting malicious traffic by calling the application programming interface of the policy execution point includes: Based on the type of the optimal defense action and the target object identifier, generate an executable instruction sequence that matches the target network policy execution point; The policy coordination engine performs atomicity verification and conflict resolution on multiple defense actions executed in parallel in the executable instruction sequence. When resource contention or logical conflict is detected between policy instructions, the execution order is dynamically adjusted based on the timing logic and network topology relationship. Priority is given to the isolation operation of key nodes in the attack path, triggering traffic redirection and IP blocking actions to ensure the consistency and execution efficiency of the defense policy. The optimized executable instruction sequence is triggered, and network status feedback data is monitored in real time. The network status feedback data includes firewall rule hit rate, isolated node resource status, and redirected traffic session characteristics. Based on the difference between the network status feedback data and the expected defense effect, the failure strategy or partial failure strategy in the executable instruction sequence is determined, and the executable instruction sequence corresponding to the optimal defense action is dynamically adjusted to regenerate an executable instruction sequence that adapts to the current attack evolution.

8. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 1, characterized in that, The triggering of network topology reconfiguration to quickly repair or isolate faulty nodes includes: When a deep reinforcement learning agent outputs an instruction that triggers topology reconstruction, or when an intrusion detection system confirms a specific threat, it analyzes the connectivity of the current network topology, identifies the edge biconnected component where the attacked node is located, and calculates the set of redundant paths that do not contain the attacked node. The network topology is dynamically adjusted based on the calculated set of redundant paths. By disconnecting malicious connections, enabling redundant links, or migrating critical services, attack paths can be cut off and network functions restored.

9. The network attack detection method based on deep reinforcement learning and topology reconstruction according to claim 1, characterized in that, After inputting the dynamic attack graph into the deep reinforcement learning agent, and analyzing the current network state to obtain the optimal defense action, the process further includes: The network topology reconstruction results and the dynamic execution results of the optimal defense actions are used as reward feedback to dynamically update the policy network of the deep reinforcement learning agent, thereby achieving continuous threat protection in a highly dynamic network environment.

10. A network attack detection device based on deep reinforcement learning and topology reconstruction, characterized in that, The device includes: The modeling module is configured to collect network traffic data, node status information and security logs in real time, dynamically model the connection relationship and communication behavior between nodes, and generate a dynamic attack graph that includes node vulnerability status, real-time attack path, attack propagation path and real-time threat indicators. The decision module is configured to input the dynamic attack graph into a deep reinforcement learning agent, analyze the current network state through the deep reinforcement learning agent, obtain the optimal defense action, generate policy instructions based on the optimal defense action, and send them to the network policy execution point; the policy execution point includes firewalls, intrusion prevention systems, and software-defined network controllers. The execution module is configured to dynamically block malicious IPs, isolate infected nodes, or redirect malicious traffic by calling the application programming interface of the policy execution point, and trigger network topology reconstruction to quickly repair or isolate faulty nodes, thereby achieving real-time detection and interception of attack behaviors.

Images