Power network attack chain dynamic deduction and intelligent response process method, system and device based on deep reinforcement learning, and medium

Abstract

The invention discloses a power network attack chain dynamic deduction and intelligent response process method, system and device based on deep reinforcement learning and a medium, and belongs to the technical field of network security and power system protection. Multi-modal data is aligned and normalized, an event view cache is constructed, and the generalization detection capability on process camouflage and memory injection attacks is improved through a federated learning collaborative detection mechanism; based on the event view cache and historical threat intelligence, generating a dynamic attack knowledge graph, constructing a deep reinforcement learning model taking the attack knowledge graph as an environment, calculating an attack influence index by using a Bayesian network, and generating a differentiated security response instruction; and realizing attack path backtracking and attack source positioning based on the attack knowledge graph. According to the method, multi-modal data fusion analysis and strategy adaptive updating are realized, and the attack chain identification accuracy and evidence chain construction integrity are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of network security and power system protection technology, specifically to a method, system, device, and medium for dynamic deduction and intelligent response of power network attack chains based on deep reinforcement learning. Background Technology

[0002] The current power network security defense system still relies mainly on static rule matching and signature recognition, which is insufficient to effectively deal with the ever-evolving new advanced threats (APTs). In particular, traditional technical means have obvious limitations when facing complex attack methods such as fileless attacks, memory injection, dynamic protocol deformation, and covert channel construction.

[0003] On the one hand, existing defense mechanisms often perform isolated analyses of single dimensions such as network traffic, host processes, or memory status, lacking cross-modal and cross-layer information fusion capabilities. This results in low accuracy in attack path deduction and delayed response strategy generation. For example, mainstream memory forensics methods are limited by insufficient snapshot mechanism integrity or lack of encryption protection, making it difficult to freeze critical memory data in real time at the initial stage of an attack. Network detection schemes based on protocol field feature matching struggle to identify encrypted or mutated attack payloads, often resulting in missed detections and false positives. Furthermore, detection, analysis, and response modules are often deployed in a dispersed manner with fragmented processes, lacking a unified and coordinated command and dispatch mechanism. This further exacerbates problems such as broken evidence chains and delayed response decisions, seriously threatening the power system's agility in perceiving and handling attack events.

[0004] On the other hand, the existing architecture also has many shortcomings in terms of secure communication and data trustworthiness assurance. Traditional hash algorithms are difficult to resist the risk of collision attacks brought about by quantum computing, and evidence storage relies on centralized databases, which are vulnerable to tampering or deletion. Key exchange mechanisms (such as RSA) are complex and lengthy, resulting in increased delays in the issuance of encryption commands, making it difficult to meet the power system's requirement for millisecond-level real-time response. At the same time, collaborative detection between cross-regional terminals often relies on plaintext parameter exchange, which is susceptible to man-in-the-middle attacks that steal model or training parameters, reducing the security, trustworthiness, and generalization ability of collaborative detection.

[0005] In summary, current power network security protection technologies struggle to achieve an effective balance between accuracy, real-time performance, and reliability. There is an urgent need for a new defense architecture with dynamic adaptability, cross-modal reasoning mechanisms, and an end-to-end trusted closed-loop structure to effectively address the increasingly complex and intelligent attack patterns of power networks and ensure the continuity, security, and stability of core business systems.

[0006] This invention constructs an end-to-end security defense system covering attack perception, path modeling, dynamic inference, and closed-loop response. Targeting threats in power networks such as fileless attacks, memory injection, and protocol tampering that are highly concealed and evolve rapidly, it achieves front-end multimodal data fusion analysis and adaptive policy updates, significantly improving the accuracy of attack chain identification and the completeness of evidence chain construction. It provides a highly reliable, low-latency dynamic security protection solution for the next-generation smart grid. Summary of the Invention

[0007] In view of the above-mentioned problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by this invention is: how to solve the problem that current power network security protection technologies are unable to achieve an effective balance between accuracy, real-time performance and reliability.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: a method for dynamic deduction and intelligent response of power network attack chains based on deep reinforcement learning, comprising: obtaining front-end multimodal data through a memory data atomic capture mechanism, process behavior baseline modeling, and collection of key protocol fields and temporal features in network traffic; aligning and normalizing the front-end multimodal data to construct an event view cache; improving the generalized detection capability against process spoofing and memory injection attacks through a federated learning collaborative detection mechanism; generating a dynamic attack knowledge graph based on the event view cache and historical threat intelligence; constructing a deep reinforcement learning model with the attack knowledge graph as the environment; calculating the attack impact index using a Bayesian network to generate differentiated security response instructions; and realizing attack path backtracking and attack source localization based on the attack knowledge graph.

[0010] As a preferred embodiment of the dynamic deduction and intelligent response process method for power network attack chains based on deep reinforcement learning described in this invention, the memory data atomic capture mechanism refers to kernel-mode atomic capture technology, combined with encrypted memory paging under hardware virtualization, recording write operation timestamps and process identifiers, and using differential snapshot technology to reduce data transmission load.

[0011] As a preferred embodiment of the dynamic inference and intelligent response process method for power network attack chains based on deep reinforcement learning described in this invention, the federated learning collaborative detection mechanism includes encrypted transmission and joint training of model parameters among multiple terminals to improve the ability to identify attack behaviors.

[0012] As a preferred embodiment of the dynamic inference and intelligent response process method for power network attack chains based on deep reinforcement learning described in this invention, the deep reinforcement learning model includes inferring attack paths based on attack knowledge graphs and generating security response strategies through temporal modeling and reasoning mechanisms.

[0013] As a preferred embodiment of the dynamic deduction and intelligent response process method for power network attack chains based on deep reinforcement learning described in this invention, the generation of a dynamic attack knowledge graph includes: integrating process call chains, memory paging records, and network traffic fingerprints from front-end multimodal data to construct graph nodes; process nodes generating call chain feature hashes through instruction sequences; memory nodes generating content hashes by monitoring changes in memory pages of key devices; network nodes extracting source IP, destination IP, port, protocol type, and temporal features to form a five-tuple fingerprint, and calculating the temporal entropy of network traffic; constructing edges between nodes through access relationships, communication interactions, and behavioral coupling relationships; process-memory edge relationships based on the number of read / write events from performance counters and memory heatmaps; process-network edge relationships based on socket binding relationships and interaction counts to calculate network-process coupling entropy indicators; threat intelligence edges establishing mapping relationships based on local data and structural features in the attack pattern library; and using graph neural networks to model the above graph, and introducing gated recurrent units to model the state change process of the attack chain on the time axis.

[0014] This preferred solution constructs graph nodes containing process call chains, memory paging records, and network fingerprints, and combines the number of read / write events, socket binding relationships, and attack pattern matching relationships to construct edge information, which can completely reconstruct the behavioral paths and coupling relationships between attack entities. By combining graph neural networks and gated recurrent units to model the graph, it is possible to realize the dynamic evolution modeling of multidimensional attack behaviors in power systems on the time axis.

[0015] As a preferred embodiment of the method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning described in this invention, the following steps are included: constructing a deep reinforcement learning model with an attack knowledge graph as the environment, including using SCADA system operating status indicators as environmental state inputs, such as device availability scores, the number of abnormal communication protocol commands, and network throughput, and introducing attack stage variables and vulnerability types as classification inputs; setting up a state space, action space, and reward function, wherein actions include isolating devices, triggering remote memory forensics, and updating access control policies, and the reward function includes positive incentives for successfully blocking attacks and reducing the probability of SCADA malfunctions, as well as penalties for incorrectly isolating legitimate devices and response delays; training and inferring attack paths through a deep reinforcement learning network, and aligning the payload sequences in memory snapshots with a dynamic time warp algorithm, matching them with historical attack paths, and converting the matching scores into reward signals to guide policy learning.

[0016] This preferred solution constructs a state space that includes the SCADA system's operating status, attack phase variables, and vulnerability types. By combining attack response actions and reward function settings, the reinforcement learning model can accurately identify the mapping relationship between different attack phases and system states. By combining a dynamic time warp algorithm to align the inferred trajectory with historical attack paths, the introduced matching score serves as reward feedback, further improving the adaptability and predictive ability of the strategy generation process to attack logic, thereby enhancing the system's dynamic response capability to multi-stage attack chains.

[0017] As a preferred embodiment of the dynamic deduction and intelligent response process method for power network attack chains based on deep reinforcement learning described in this invention, the generation of differentiated security response instructions includes: generating a set of response actions based on the attack chain deduction results; constructing a Bayesian network model by combining operational status parameters collected by the SCADA system, including CPU utilization, number of abnormal memory writes, and network traffic change rate; performing posterior inference on candidate response actions; calculating the attack impact index based on the probability distribution of each action on changes in key indicators; combining resource constraints such as CPU utilization limit, response action concurrency, and network bandwidth load; and using a greedy algorithm to select the set of response actions with the smallest impact index; and using an approximate inference algorithm to iterate in the Bayesian network using inter-node message passing, limiting the maximum number of iterations or the termination condition of the change magnitude, to generate the final security response instruction that meets the millisecond-level response latency requirement.

[0018] This preferred solution constructs a Bayesian network to infer the posterior impact probability of response actions. Combined with the current operating status of the system and resource constraints, it can achieve a quantitative assessment of the risks and benefits of different response actions. A greedy algorithm is used to select the set of response actions with the smallest impact index, and an approximate calculation method is introduced in the inference process to ensure that the millisecond-level decision response requirements are met while ensuring accuracy. This improves the system's security handling efficiency and practicality in scenarios with rapidly changing attack situations.

[0019] This invention provides a system for dynamic simulation and intelligent response of power network attack chains based on deep reinforcement learning.

[0020] To address the aforementioned technical problems, this invention provides the following technical solution: a power network attack chain dynamic deduction and intelligent response process system based on deep reinforcement learning, comprising: a front-end intelligent perception and front-end multimodal data acquisition module, a cloud-based attack chain dynamic deduction and intelligent decision-making module, and a multimodal source tracing analysis and closed-loop response module; the front-end intelligent perception and front-end multimodal data acquisition module is used to obtain front-end multimodal data through a memory data atomic capture mechanism, process behavior baseline modeling, and the acquisition of key protocol fields and temporal features in network traffic; the front-end multimodal data is aligned and normalized to construct an event view cache, and the generalized detection capability against process spoofing and memory injection attacks is improved through a federated learning collaborative detection mechanism; the cloud-based attack chain dynamic deduction and intelligent decision-making module is used to generate a dynamic attack knowledge graph based on the event view cache and historical threat intelligence, construct a deep reinforcement learning model with the attack knowledge graph as the environment, calculate the attack impact index using a Bayesian network, and generate differentiated security response instructions; the multimodal source tracing analysis and closed-loop response module is used to realize attack path backtracking and attack source localization based on the attack knowledge graph.

[0021] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the described method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning.

[0022] The present invention provides a computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning.

[0023] The beneficial effects of this invention are as follows: This invention significantly enhances the ability of power grids to resist novel attacks that are highly concealed and evolve rapidly. Its beneficial effects are mainly reflected in the following aspects: By using deep reinforcement learning to infer the attack evolution path in real time, and combining it with a dynamic graph neural network to construct an attack knowledge graph that fuses multimodal data at the front end, accurate prediction and adaptive optimization of attack paths are achieved; Memory atomic capture, differential snapshots, and federated learning techniques are employed to reduce data transmission volume while ensuring the security and privacy of multi-source data, and quantum encryption and the QUIC protocol are used to solve the high latency problem of traditional encrypted transmission, meeting the millisecond-level response requirements of power systems; A dynamic time warp algorithm is integrated to reverse-analyze the attack payload logic, and a Bayesian network and greedy algorithm are combined to generate differentiated response strategies, effectively balancing security protection strength and system reliability; Finally, a full-process defense system is formed from attack perception, dynamic inference, intelligent decision-making to closed-loop response, solving the problems of broken evidence chains and delayed decision-making caused by isolated data and fragmented responses in traditional methods, providing a highly accurate, low-latency, and end-to-end reliable security protection solution for smart grids. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 The above is a flowchart of a method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning, which is provided as an embodiment of the present invention.

[0026] Figure 2 This is a flowchart of a method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning, provided as an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of a scheme for a dynamic simulation and intelligent response process system for power network attack chains based on deep reinforcement learning, provided as an embodiment of the present invention. Detailed Implementation

[0028] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0029] Example 1, referring to Figure 1 and Figure 2 This is one embodiment of the present invention, which provides a method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning, including:

[0030] S1. Front-end multimodal data is obtained through memory data atomic capture mechanism, process behavior baseline modeling, and collection of key protocol fields and timing features in network traffic.

[0031] S2. Align and normalize the multimodal data from the front end, build an event view cache, and improve the generalized detection capability against process spoofing and memory injection attacks through a federated learning collaborative detection mechanism.

[0032] S3. Based on event view cache and historical threat intelligence, generate a dynamic attack knowledge graph, construct a deep reinforcement learning model with the attack knowledge graph as the environment, use Bayesian network to calculate the attack impact index, and generate differentiated security response instructions.

[0033] S4. Achieve attack path tracing and attack source location based on attack knowledge graph.

[0034] like Figure 1 and Figure 2 As shown, Figure 1 This is the overall flowchart of the present invention. Figure 2 This is a detailed flowchart of the present invention.

[0035] It should be noted that during the operation of power systems, terminal nodes face complex and ever-changing attack threats, such as process spoofing, memory injection, and abnormal communication behaviors. Traditional security detection methods struggle to achieve accurate identification without exposing data, and their deduction and response to attack paths often suffer from lag and locality issues. This invention introduces a front-end multimodal data acquisition mechanism based on memory behavior, process activity, and network fingerprints, and performs alignment and normalization processing to construct a unified event view cache, serving as the core foundation for collaborative detection and graph modeling. Based on this, a federated learning-based behavioral baseline model and a deep reinforcement learning deduction model are constructed, combined with an attack knowledge graph and a Bayesian network decision-making mechanism, to achieve real-time detection, path prediction, and differentiated response command generation for complex attack chains, effectively improving the security protection capabilities of power networks in highly dynamic environments.

[0036] Example 2, an embodiment of the present invention, provides a method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning, based on the previous embodiment, including:

[0037] In the embodiments of this application, in step S1, front-end multimodal data is obtained through a memory data atomic capture mechanism, process behavior baseline modeling, and collection of key protocol fields and timing features in network traffic.

[0038] Atomic capture of memory data: It adopts kernel-mode atomic capture technology, combined with hardware virtualization encrypted memory paging, records write operation timestamps and process IDs, and reduces the amount of data transfer through differential snapshot technology.

[0039] Process behavior baseline modeling: eBPF (Linux) and WFP (Windows) cross-platform monitoring framework, constructing a family tree graph of power business processes, and combining Control-flow Enforcement Technology (CET) to enhance process control flow integrity protection and prevent malicious instruction hijacking from causing abnormal memory access.

[0040] Network traffic dynamic fingerprint generation: Based on the Data Plane Development Kit (DPDK) engine, 10Gbps-level traffic capture is achieved, DNP3 / Modbus protocol payloads are deeply analyzed, and Long Short-Term Memory (LSTM) network detection is used to generate a five-tuple dynamic traffic fingerprint (source IP, destination IP, port, protocol type, and timing characteristics).

[0041] Specifically, the memory data atomicity capture unit, based on the kernel-mode atomicity monitoring mechanism and combined with encrypted memory paging technology under hardware virtualization, achieves high-granularity awareness of write operations. It monitors write operation events in real time by calling the kernel memory access interface and accurately records the timestamp and process ID of each operation. To reduce transmission load, a differential snapshot mechanism is introduced, synchronizing only the changed data segments, and finally writing them to the memory data buffer through the memory mapping interface, ensuring efficient and complete data transmission.

[0042] The process behavior baseline modeling unit adopts a cross-platform monitoring framework. On Linux systems, it uses the Extended Berkeley Packet Filter (eBPF), and on Windows systems, it uses the Windows Filtering Platform (WFP). Process running status is collected in real time through the system's native command interface. The system constructs a process family tree graph to depict the parent-child relationships and behavioral paths between processes, and introduces a CET mechanism to strengthen control flow integrity protection, effectively identifying abnormal control flow instructions injected through techniques such as ROP, and indirectly blocking potential memory hijacking behaviors. Monitoring is particularly enhanced to address the risks of malicious process injection and instruction tampering that may occur in power systems. Finally, behavioral data such as process ID, name, and parameters are collected to generate a process behavior baseline model, which is then written to the local cache.

[0043] The network traffic dynamic fingerprint generation unit deploys monitoring probes on key links in the network protocol stack (such as between the network driver layer and the IP layer) and achieves high-speed packet capture based on DPDK. The system embeds an LSTM model to model the temporal characteristics of network traffic, accurately identify potential hidden channels (such as data reload fields, abnormal packet timing, etc.), and performs in-depth analysis of key protocol content to extract information such as source / destination IP, port, protocol type, and inter-packet time interval, forming a five-tuple traffic fingerprint, which is then stored in the network traffic fingerprint cache.

[0044] In this embodiment of the application, step S2 aligns and normalizes the front-end multimodal data, constructs an event view cache, and improves the generalized detection capability against process spoofing and memory injection attacks through a federated learning collaborative detection mechanism.

[0045] The federated learning collaborative detection mechanism includes encrypted transmission and joint training of model parameters across multiple terminals, which enhances the ability to identify attack behaviors.

[0046] Multimodal feature fusion and caching: Align and normalize the dynamic features of memory behavior, process activity and network traffic to build a unified event view cache to support subsequent graph modeling and federated training.

[0047] In one optional implementation, the alignment and normalization process includes mapping the collected memory write operation behavior, process running status information and five-tuple network traffic characteristics to a unified time series window, aligning them with a unified timestamp, and using Z-Score normalization to normalize the continuous values ​​such as duration, access frequency and interaction interval in each modality data to ensure that the three types of behavioral features are comparable when input into the event view.

[0048] In another optional implementation, the alignment and normalization process includes constructing an index key table by mapping process IDs to network ports, cross-matching memory paging events, process call paths and traffic features based on the index association method, and using the Min-Max normalization method to compress various numerical features to a uniform range [0,1] to construct node feature vectors in the event view cache.

[0049] This invention, through alignment and normalization processing, can construct a unified data representation structure even when there are differences in temporal granularity and inconsistent data dimensions in different modal data, effectively improving the input consistency and training stability of subsequent behavior modeling and attack path inference.

[0050] Federated learning collaborative detection: Cross-regional terminals protect model parameters using a post-quantum encryption algorithm recommended by NIST, and achieve efficient and secure transmission via the Quick UDP Internet Connection (QUIC) protocol. Collaborative training of the behavioral baseline model improves the generalized detection capability against process spoofing and memory injection attacks.

[0051] Specifically, the federated learning collaborative detection unit employs a post-quantum encryption algorithm recommended by NIST to protect model parameters. Combined with the QUIC protocol based on Transport Layer Security (TLS), it constructs an efficient, low-latency parameter synchronization channel, ensuring the security and real-time performance of model interaction between power terminals. Each terminal conducts distributed collaborative training based on locally collected memory, process, and network multimodal behavior data, effectively improving the model's generalization detection capability against advanced persistent attacks such as process spoofing and memory injection. Finally, the local model update results and detection outputs are synchronized to the federated model cache, providing support for subsequent centralized decision-making.

[0052] In one optional implementation, federated learning collaborative detection includes each terminal node independently training an initial behavioral baseline model based on locally collected front-end multimodal data, encrypting the model parameters using a post-quantum encryption algorithm recommended by NIST, and synchronizing them to the cloud through a QUIC protocol channel built on TLS. The cloud service then averages and aggregates the model update parameters uploaded by all terminals before distributing them to achieve synchronized updates of the global model.

[0053] In another optional implementation, federated learning collaborative detection includes introducing an attack spoofing sample enhancement mechanism during the local training process of each terminal node, inserting low-frequency high-entropy instruction sequences into the process behavior trajectory to simulate advanced persistent attack scenarios, and sending the updated model parameters to neighboring terminals through the QUIC protocol via a group communication strategy after training to perform local model exchange and build a decentralized distributed federated learning system.

[0054] This invention, through model training collaboration and parameter transmission strategies, can improve the generalization detection capability of the behavioral baseline model for complex attack behaviors such as process spoofing and memory injection, while ensuring that terminal data privacy is not leaked, and enhance the model's adaptability to the dynamic behavior patterns of multiple terminals in the heterogeneous environment of the power system.

[0055] In the embodiments of this application, in step S3, a dynamic attack knowledge graph is generated based on the event view cache and historical threat intelligence, a deep reinforcement learning model with the attack knowledge graph as the environment is constructed, an attack impact index is calculated using a Bayesian network, and differentiated security response instructions are generated.

[0056] The deep reinforcement learning model infers attack paths based on attack knowledge graphs and generates security response strategies through temporal modeling and reasoning mechanisms.

[0057] Generating a dynamic attack knowledge graph involves integrating process call chains, memory paging records, and network traffic fingerprints from front-end multimodal data to construct graph nodes. Process nodes generate call chain feature hashes through instruction sequences, memory nodes generate content hashes by monitoring changes in memory pages of key devices, and network nodes extract source IP, destination IP, port, protocol type, and temporal features to form a five-tuple fingerprint and calculate the temporal entropy of network traffic. Edges between nodes are constructed through access relationships, communication interactions, and behavioral coupling relationships. The process-memory edge relationship is based on the number of read / write events from performance counters and memory heatmaps, while the process-network edge relationship is based on socket binding relationships and interaction counts to calculate the network-process coupling entropy index. Threat intelligence edges are based on local data and structural features in the attack pattern library to establish mapping relationships. A graph neural network is used to model the above graph, and a gated recurrent unit is introduced to model the state change process of the attack chain on the time axis.

[0058] The deep reinforcement learning model, built around an attack knowledge graph, uses SCADA system operational status indicators as environmental inputs, including device availability scores, the number of abnormal communication protocol commands, and network throughput. Attack phase variables and vulnerability types are introduced as classification inputs. A state space, action space, and reward function are defined. Actions include isolating devices, triggering remote memory forensics, and updating access control policies. The reward function includes positive incentives for successfully blocking attacks and reducing the probability of SCADA malfunctions, as well as penalties for incorrectly isolating legitimate devices and response delays. Attack paths are inferred through deep reinforcement learning network training, and a dynamic time warp algorithm is used to align payload sequences in memory snapshots with historical attack paths. The matching scores are then converted into reward signals to guide policy learning.

[0059] Generating differentiated security response instructions involves generating a set of response actions based on attack chain deduction results, constructing a Bayesian network model by combining operational status parameters collected by the SCADA system, including CPU utilization, number of abnormal memory writes, and network traffic change rate, and performing posterior inference on candidate response actions; calculating the attack impact index based on the probability distribution of each action on changes in key indicators, and using a greedy algorithm to select the set of response actions with the smallest impact index, taking into account resource constraints such as CPU utilization limit, response action concurrency, and network bandwidth load; and using an approximate inference algorithm to iterate in the Bayesian network using inter-node message passing, limiting the maximum number of iterations or the termination condition of the change magnitude, to generate the final security response instruction that meets the millisecond-level response latency requirement.

[0060] Attack knowledge graph construction: Integrate front-end multimodal data (process call chain, memory paging record, network fingerprint) with historical threat intelligence, and use graph neural network (GNN) to model attack entity relationships to generate a dynamic knowledge graph.

[0061] Dynamic Deduction through Deep Reinforcement Learning: A Deep Reinforcement Learning (DRL) model is constructed using an attack knowledge graph as its environment. The state space (system state, device load), action space (isolation, forensics, policy update), and reward function (minimizing the attack impact exponent) are defined. Attack path prediction is dynamically optimized using Q-Learning, and the attack payload logic is reverse-analyzed using the Dynamic Time Warping (DTW) algorithm.

[0062] Real-time Bayesian Network Decision Making: Based on Bayesian networks, the attack impact index is calculated, and differentiated security response instructions are output through a pipeline decision-making process of "feature matching → impact assessment → resource scheduling".

[0063] Specifically, attack chain modeling, path deduction, and response strategy generation are performed on multimodal data collected from the front end in the cloud. The core steps include attack knowledge graph construction, dynamic graph neural network modeling, temporal feature learning, and dynamic graph update, forming an efficient perception, accurate prediction, and intelligent response capability for power system attack behavior.

[0064] In terms of attack knowledge graph construction, we first integrate multimodal data such as process call chains, memory paging records, and network fingerprints uploaded from the front end, and fuse them with a historical threat intelligence database to construct an attack knowledge graph for inference. Graph nodes include processes, memory pages, and network communication endpoints, while graph edges reflect the access, communication, and behavioral coupling relationships between entities.

[0065] S3.1.1 Define the nodes.

[0066] The process node extracts the process ID through the operating system interface and calculates its call chain feature hash to represent the execution path. To meet real-time requirements, the hash calculation uses the SHA3-256 hardware acceleration instruction under the ARMv9 architecture, constructed as follows:

[0067] H(S)=SHA3_HW(s1⊕s2⊕…⊕s n )

[0068] Here, S represents the process call chain instruction sequence, and ⊕ indicates a bitwise XOR operation. Furthermore, the CPU and memory resource utilization of the corresponding process is recorded as dynamic behavioral characteristics.

[0069] Memory node monitoring tracks core memory page changes in critical equipment (such as protection devices and control terminals), recording write operation timestamps t. write And use SHA3-256 to generate the content hash:

[0070] H mem (M) = SHA3-256(M)

[0071] Where M represents the content of a memory page, used to detect memory tampering behavior such as ASLR bypass.

[0072] Network nodes are constructed from traffic data captured by eBPF, and a five-tuple fingerprint consisting of source IP, destination IP, port, protocol, and timing is extracted. The system further calculates the timing entropy of this flow to assess the regularity of communication, using the following formula:

[0073]

[0074] Where, p i It represents the probability distribution of network state over time, and is suitable for distinguishing between periodic SCADA traffic and abnormal burst behavior.

[0075] S3.1.2 Define edge relationships.

[0076] In edge relationship modeling, the edge relationship between processes and memory is based on the number N of read / write events captured by the Intel PMU performance counter. rw Edge features are generated by combining memory access heatmaps; and a Cuckoo Filter structure is used to efficiently compress paging paths, adapting to the storage limitations of power grid edge nodes. The edge relationships between processes and the network are determined through Socket binding information and the number of protocol interactions N. proto The system extracts and constructs a "network-process coupling entropy" metric to measure the synchronization of computation and communication behaviors:

[0077]

[0078] A dynamic threshold setting C > 0.65 triggers an alert for abnormal behavior coupling. The threat intelligence side algorithm matches local data with structural features in the attack pattern library to establish an attack pattern mapping relationship, enhancing the prior representation capability of the attack pattern graph.

[0079] S3.1.3 Dynamic graph neural network modeling.

[0080] In the graph modeling stage, a Dynamic Graph Attention Network (DGAT) structure is introduced to model the aforementioned graph. The dimension of the node input feature vector is set to 256, and the edge weights are assigned values ​​after normalization based on the interaction frequency. DGAT uses a dynamic multi-head attention mechanism to adjust the attention focus range: 4 heads are used when the node degree is less than 50, 8 heads are used when the degree is between 50 and 200, and 12 heads are used when the degree is greater than 200 to enhance global perception. Each attention head focuses on the node feature h. i Calculate the attention score after performing a linear mapping:

[0081]

[0082] Among them, W k Let a be the weight matrix of the k-th head. k These are learnable parameters.

[0083] To capture the temporal evolution characteristics of the attack chain, a gated recurrent unit (GRU) is introduced to process the state trajectories of nodes and edges that change over time. The GRU updates the gate z... t Reset the door r t and candidate hidden state Automatically model the migration path of attack behavior on the timeline, such as the continuous changes in state during the process from initial intrusion to lateral movement.

[0084] S3.1.4 Knowledge graph is dynamically updated.

[0085] In the dynamic simulation and intelligent response system for power grid attack chains, the system receives real-time model feedback data from front-end devices through a federated learning mechanism and uses a post-quantum encryption algorithm recommended by NIST to ensure data security during transmission. Data is transmitted via the QUIC protocol and, based on an asynchronous update mechanism and prefetching strategy, performs incremental graph updates every minute, while ensuring that the response latency for core node updates is controlled within 10ms. For threat intelligence fusion, the system introduces a hybrid subgraph matching strategy: for known attack patterns, it uses the CUDA-Graph engine, which supports fuzzy matching of regular expressions; for unknown patterns, it uses GraphSAGE to calculate the cosine distance between structural similarity and node embeddings. By continuously iteratively updating node attributes and edge relationships, the system significantly improves the accuracy and response speed for identifying attack chain variants.

[0086] The dynamic deduction steps of deep reinforcement learning are as follows:

[0087] S3.2.1 Environmental perception input.

[0088] In the power network security scenario, the environmental state input of the deep reinforcement learning model consists of key operational indicators of the SCADA system, comprehensively reflecting the current operational status of the system. Among these, the equipment availability score A∈[0,1] represents the overall health of the current system equipment; the number of abnormal communication protocol commands N... proto-abn Network throughput T was obtained through statistical comparison with the whitelist instruction set. net This represents the effective data traffic per unit time, collected by the front-end network monitoring probe. Furthermore, to incorporate attack context information, the system encodes the current attack phase as the attack phase variable S. attack (e.g., initial intrusion is 1, privilege escalation is 2, lateral movement is 3, etc.), and the exploit type V is... used The mapping is in the form of one-hot encoding, which serves as the input feature for classification.

[0089] S3.2.2, Reward Function.

[0090] In the context of power network security protection, the reward function is a core design element in reinforcement learning strategy optimization, guiding the model to select appropriate response actions under different attack scenarios. The system categorizes reward functions into two types: positive incentives and negative penalties.

[0091] In terms of positive rewards, successfully blocking an attack will grant a reward of R. b =+1; If the probability of SCADA malfunction can be reduced, the reward is P. r =P original -P new , where P original With P new These represent the probabilities of erroneous actions before and after the response, respectively. Regarding negative rewards, if a legitimate device is incorrectly isolated, a penalty of R is applied. mis = -1; if the response delay D exceeds the threshold, a penalty R is applied. delay = -0.3. The final total reward value is:

[0092] R = R b +R r +R mis +R delay

[0093] S3.2.3 DRL Model Training and Inference.

[0094] The Actor network employs a 3-layer Multilayer Perceptron (MLP) architecture with a 256-128-64 structure, receiving data containing the operating states of the power system. Let the input state vector be s, the first-layer weight matrix be W1, and the bias be b1. Then the first-layer output is:

[0095] h1 = ReLU(W 1s +b1)

[0096] Similarly, the second layer output:

[0097] h2 = ReLU(W2h1 + b2)

[0098] The probability distribution of the third-layer output action is obtained by Softmax(W3h2+b3), which guides the system to select defensive actions in power grid attack scenarios.

[0099] The Critic network is also a 3-layer MLP with a 256-128-1 structure, used to output the state value function V(s), and its computation process is similar to that of the Actor network. The model uses Generalized Advantage Estimation (GAE) to calculate the advantage function A. t Combined with instant rewards t The policy gradient update is achieved by using the difference between the state value and the policy value, which improves the long-term policy stability and adapts to the dynamic changes of power network attacks.

[0100] To enhance the modeling capability of attack logic sequences, the system introduces the DTW algorithm to align the payload sequence in the memory snapshot. Let the actual payload sequence be X = [x1, x2, ..., x...]. m The reference template is Y = [y1, y2, ..., y]. m The local distance matrix is ​​d(i,j)=|x i -y i The cumulative distance of DTW is:

[0101]

[0102] Align the sequences accordingly and extract logical features. Calculate the matching score S. DTW Convert into an additional reward signal and update the original reward as follows:

[0103] R new =R+αS DTW

[0104] Here, α is an adjustment coefficient used to guide the DRL model to learn attacker behavior patterns, which improves the accuracy of power network attack path prediction and the effectiveness of defense strategies, providing strong support for dealing with complex and ever-changing power network attacks.

[0105] S3.2.4 Online simulation and strategy optimization.

[0106] During the real-time defense phase of the power grid, the system collects the power grid's operating status every 100ms and inputs it into the Actor network to generate the current optimal defense action. In the action selection process, an ∈-greedy strategy is adopted to balance strategy utilization and exploration: that is, the optimal action under the current strategy is selected with a probability of 1-∈, and an action is randomly selected with a probability of ∈=0.1, ensuring that the system has a certain degree of adaptability and self-learning ability when facing new threats.

[0107] Real-time decision making using Bayesian networks:

[0108] In the power network security response system, deep reinforcement learning models can deduce the stage structure of the current attack chain, such as the pattern of "initial intrusion → memory injection → lateral movement". Combined with operational status parameters (such as CPU utilization, number of abnormal memory writes, network traffic change rate, etc.) collected by the Supervisory Control and Data Acquisition (SCADA) system, the system maps the attack chain status and device operational information to a predefined response rule base, automatically generating a set of candidate response actions. Typical response actions include: isolating the communication ports of a certain type of device, triggering the memory forensics module of a remote device, and updating the access control policy (ACL) for a specific area.

[0109] To assess the potential impact of each candidate action on system security and reliability, a Bayesian network model is constructed, where each node represents a system state variable or action event, and edges represent causal dependencies. Posterior inference is used to calculate the probability distribution of changes in key indicators caused by each candidate action, and its expected impact value is quantified. The impact scoring function is defined as:

[0110]

[0111] Quantify the potential risks this action poses to system reliability.

[0112] During the action decision-making phase, the system employs a greedy algorithm under resource constraints, selecting the set of actions with the minimum expected impact while ensuring safety. Resource constraints include CPU utilization limits, response action concurrency limits, and network instruction bandwidth load, preventing resource contention or scheduling blockage caused by excessive response operations. This mechanism effectively avoids secondary risks arising from policy overfitting or response overload, ensuring the continuous and stable operation of the power system.

[0113] To meet the stringent requirements of millisecond-level response latency in power grid network security systems, the system employs the LoopyBeliefPropagation (LBP) approximate inference algorithm instead of exact Bayesian inference. LBP rapidly converges to an approximate posterior distribution through iterative message passing between nodes. The system is limited to a maximum of 10 iterations, or terminates early if the changes in two consecutive iterations fall below a threshold ε.

[0114] In summary, a closed-loop response path was achieved, encompassing attack chain deduction, action generation, impact assessment, and optimal selection. This integrated key components such as deep reinforcement learning, rule base mapping, Bayesian network inference, and resource-aware scheduling. Combined with the efficient inference capabilities provided by LoopyBeliefPropagation, the system constructed a fusion-based, high-real-time attack response decision-making mechanism capable of achieving rapid, accurate, and resource-controlled intelligent responses to complex and continuously evolving power grid attack scenarios.

[0115] DRL-GNN-DTW collaborative mechanism:

[0116] To address the multiple challenges posed by the rapid evolution of attack chains, complex paths, and stringent response requirements in power grids, this system designs and implements a collaborative inference and decision-making mechanism comprised of a Graph Neural Network (GNN), a Deep Reinforcement Learning (DRL) algorithm, and a Dynamic Time Warp (DTW) algorithm. These three components respectively undertake the key tasks of modeling, inference, and verification, and are linked together through a structured data flow interface to construct an end-to-end, dynamically adaptable power grid attack response strategy generation system.

[0117] First, the GNN receives multimodal input from the front-end perception, including process call chains, memory paging information, and network traffic fingerprints. Combined with historical threat intelligence, it dynamically constructs an attack knowledge graph. Nodes represent entities in the system such as processes, memory pages, and network ports, while edges represent behavioral relationships such as access, communication, and invocation. The GNN generates a state vector S by aggregating neighbor node information. GNN This vector comprehensively reflects the abnormal behavior evolution characteristics of each key component in the power system and serves as the environmental state input for the DRL model.

[0118] Secondly, DRL uses the state vector S output by the GNN. GNN Together with the operating status of the power SCADA system (such as equipment load, voltage fluctuations, abnormal communication commands, etc.), they constitute a complete environmental state S Total The data is then input into a decision network based on an Actor-Critic architecture. In each round of inference, the DRL model outputs the optimal response action 'a'. t This includes actions such as isolating suspicious devices, triggering specific processes for forensic analysis, or adjusting access control policies, while simultaneously receiving real-time feedback signals R from the environment. tThis is used to assess the actual impact of the action on system security and guide the next round of policy updates.

[0119] To verify whether the attack paths predicted by DRL are consistent with actual attack behavior and to guide it in learning strategies with more attack semantics, this system introduces the DTW algorithm as a verification mechanism. After the DRL outputs the action sequence, the system records its execution trajectory T on the timeline. pred And compared with typical historical attack paths T in the knowledge base ref Matching is performed by calculating the DTW distance D. DTW This is used to quantify the similarity between the two in terms of behavioral logic. This similarity score is then incorporated into the reward function, modifying the original reward as follows:

[0120] R new =R+α*(1-D DTW )

[0121] It effectively improves the fit and generalization ability of attack chain logic in DRL policy learning.

[0122] In the embodiments of this application, step S4 is based on the attack knowledge graph to realize attack path backtracking and attack source location.

[0123] Cross-modal intelligent tracing: The Transformer model with domain knowledge annotation is used for protocol semantic modeling, and the SCADA state machine system state machine is further constructed. Attack path backtracking and attack source location are realized based on GNN knowledge graph.

[0124] Specifically, the attack path is reversed by fusing multimodal data with GNN knowledge graph, abnormal nodes are filtered by graph attention network, the attack path is backtracked by bidirectional random walk algorithm, and the logical consistency of the attack path is verified by dynamic time warp (DTW) algorithm.

[0125] Example 3, referring to Figure 3 This is one embodiment of the present invention, which provides a dynamic inference and intelligent response process system for power network attack chains based on deep reinforcement learning, including a front-end intelligent perception and front-end multimodal data acquisition module, a cloud-based attack chain dynamic inference and intelligent decision-making module, and a multimodal source tracing analysis and closed-loop response module.

[0126] The front-end intelligent perception and front-end multimodal data acquisition module is used to obtain front-end multimodal data through the atomic capture mechanism of memory data, process behavior baseline modeling, and collection of key protocol fields and time-series features in network traffic; the front-end multimodal data is aligned and normalized, an event view cache is built, and the generalized detection capability against process spoofing and memory injection attacks is improved through a federated learning collaborative detection mechanism;

[0127] The cloud-based attack chain dynamic simulation and intelligent decision-making module is used to generate a dynamic attack knowledge graph based on event view cache and historical threat intelligence, build a deep reinforcement learning model with the attack knowledge graph as the environment, use Bayesian network to calculate the attack impact index, and generate differentiated security response instructions.

[0128] The multimodal source tracing analysis and closed-loop response module is used to achieve attack path backtracking and attack source location based on attack knowledge graph.

[0129] This embodiment also provides an electronic device applicable to a method for dynamic deduction and intelligent response of power network attack chains based on deep reinforcement learning, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for dynamic deduction and intelligent response of power network attack chains based on deep reinforcement learning as proposed in the above embodiment.

[0130] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning, as proposed in the above embodiment.

[0131] The storage medium proposed in this embodiment and the method for implementing a dynamic deduction and intelligent response process of power network attack chain based on deep reinforcement learning proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0132] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0133] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning, characterized in that: include, Multimodal front-end data is obtained through in-memory data atomic capture mechanism, process behavior baseline modeling, and collection of key protocol fields and timing features in network traffic; The front-end multimodal data is aligned and normalized, an event view cache is built, and a federated learning collaborative detection mechanism is used to improve the generalized detection capability against process spoofing and memory injection attacks. Based on event view caching and historical threat intelligence, a dynamic attack knowledge graph is generated, a deep reinforcement learning model with the attack knowledge graph as the environment is constructed, and a Bayesian network is used to calculate the attack impact index to generate differentiated security response instructions. Attack path tracing and attack source location are achieved based on attack knowledge graphs.

2. The method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning as described in claim 1, characterized in that: The memory data atomic capture mechanism refers to kernel-mode atomic capture technology, which combines encrypted memory paging under hardware virtualization, records write operation timestamps and process identifiers, and uses differential snapshot technology to reduce data transmission load.

3. The method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning as described in claim 2, characterized in that: The federated learning collaborative detection mechanism includes encrypted transmission and joint training of model parameters between multiple terminals to improve the ability to identify attack behaviors.

4. The method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning as described in claim 3, characterized in that: The deep reinforcement learning model includes inferring attack paths based on attack knowledge graphs and generating security response strategies through temporal modeling and reasoning mechanisms.

5. The method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning as described in claim 4, characterized in that: The generation of the dynamic attack knowledge graph includes... The process call chain, memory paging record and network traffic fingerprint in the front-end multimodal data are integrated to construct graph nodes. The process node generates call chain feature hash through instruction sequence, the memory node generates content hash by monitoring the memory page changes of key devices, and the network node extracts source IP, destination IP, port, protocol type and time series features to form a five-tuple fingerprint and calculates the time series entropy of network traffic. Edges between nodes are constructed through access relationships, communication interactions, and behavioral coupling relationships. Edges between processes and memory are based on the number of read and write events from performance counters and memory heatmaps. Edges between processes and networks are based on the calculation of network-process coupling entropy indicators based on socket binding relationships and interaction counts. Threat intelligence edges are based on establishing mapping relationships between local data and structural features in the attack pattern library. A graph neural network is used to model the above graph, and a gated recurrent unit is introduced to model the state change process of the attack chain on the time axis.

6. The method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning as described in claim 4, characterized in that: The construction of a deep reinforcement learning model based on attacking knowledge graphs includes, The SCADA system's operational status indicators are used as environmental status inputs, including device availability scores, the number of abnormal communication protocol commands, and network throughput. Attack phase variables and vulnerability types are also introduced as classification inputs. The system sets up a state space, an action space, and a reward function. Actions include isolating devices, triggering remote memory forensics, and updating access control policies. Reward functions include positive incentives for successfully blocking attacks and reducing the probability of SCADA malfunctions, as well as penalties for incorrectly isolating legitimate devices and response delays. Attack paths are inferred by training a deep reinforcement learning network, and the payload sequence in the memory snapshot is aligned with the dynamic time warp algorithm and matched with historical attack paths. The matching score is converted into a reward signal to guide policy learning.

7. The method for dynamic inference and intelligent response of power network attack chains based on deep reinforcement learning as described in claim 4, characterized in that: The generation of differentiated security response instructions includes, Based on the attack chain inference results, a set of response actions is generated. Combined with the operational status parameters collected by the SCADA system, including CPU utilization, number of abnormal memory writes and network traffic change rate, a Bayesian network model is constructed to perform posterior inference on candidate response actions. The attack impact index is calculated based on the probability distribution of each action on the changes in key indicators. Combining the resource constraints of CPU utilization limit, response action concurrency and network bandwidth load, a greedy algorithm is used to select the set of response actions with the smallest impact index. An approximate inference algorithm is used to iterate in a Bayesian network using inter-node message passing, with a limit on the maximum number of iterations or a termination condition based on the magnitude of change, to generate a final secure response instruction that meets the millisecond-level response latency requirement.

8. A system for dynamic deduction and intelligent response of power network attack chains based on deep reinforcement learning, employing the method for dynamic deduction and intelligent response of power network attack chains based on deep reinforcement learning as described in any one of claims 1 to 7, characterized in that, include: Front-end intelligent perception and front-end multimodal data acquisition module, cloud-based attack chain dynamic simulation and intelligent decision-making module, and multimodal source tracing analysis and closed-loop response; The front-end intelligent perception and front-end multimodal data acquisition module is used to obtain front-end multimodal data through a memory data atomic capture mechanism, process behavior baseline modeling, and the acquisition of key protocol fields and temporal features in network traffic; the front-end multimodal data is aligned and normalized, an event view cache is constructed, and the generalized detection capability against process spoofing and memory injection attacks is improved through a federated learning collaborative detection mechanism; The cloud-based attack chain dynamic simulation and intelligent decision-making module is used to generate a dynamic attack knowledge graph based on event view cache and historical threat intelligence, construct a deep reinforcement learning model with the attack knowledge graph as the environment, use Bayesian network to calculate the attack impact index, and generate differentiated security response instructions. The multimodal source tracing analysis and closed-loop response module is used to achieve attack path backtracking and attack source location based on the attack knowledge graph.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for dynamic inference and intelligent response process of power network attack chain based on deep reinforcement learning, as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for dynamic inference and intelligent response of power network attack chain based on deep reinforcement learning, as described in any one of claims 1 to 7.

Images