Power system honey dynamic deployment defense method based on learning perception

Abstract

The invention discloses a power system honey dynamic deployment defense method based on learning perception, and belongs to the technical field of network security. A traditional honey array static deployment method is insufficient in information acquisition and dynamic change ability, and the power system honey array dynamic deployment defense method based on learning perception disclosed by the invention performs intelligent interaction with an attacker based on a message passing neural network to improve the security threat perception ability. And a honey array dynamic deployment mechanism is established according to a depth deterministic strategy gradient algorithm to enhance the fraudulence of the simulation power system, and finally, the overall defense capability and the intelligent level of the network defense system are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a learning-based perception-based power system honey array dynamic deployment defense method. Background Art

[0002] In the process of deep integration of digitalization, intelligence and electrification, modern power systems face multiple cybersecurity challenges, including dealing with the extremely rich attack tools and technical means of advanced persistent threat attackers. These tools may include custom malware, zero-day vulnerability exploits, and carefully designed backdoor programs, which directly affect the production, transmission and distribution of electricity. Traditional passive defense measures such as firewalls and intrusion detection systems are difficult to fully cope with, so security experts have proposed an active defense strategy such as honeypots.

[0003] Honeypots simulate the vulnerabilities and resources of real systems to attract the attention of attackers, and then monitor attack behaviors, analyze attack methods, and obtain attack patterns, thereby enhancing the ability to understand potential threats and improving the effectiveness of network defense. The main purpose of honeypots is not to protect the system directly, but to indirectly deceive attackers to attack the simulated system and take corresponding defense measures in advance to avoid security threats to the real system. Honey arrays are a network composed of multiple honeypots. They simulate a more complex, distributed, scenario-based simulation environment to attract and confuse more advanced attackers. Honey arrays are not a simple combination of multiple honeypots. It needs to coordinate multiple honeypots to create a complete virtual network environment to more comprehensively analyze the behavior and intentions of attackers. In the power system, the information is complex, the network interfaces are numerous and difficult to locate and trace, and attackers customize or use special network weapons by collecting detailed information of the attack targets for a long time to carry out targeted and precise attacks. The traditional static deployment method of honey arrays cannot capture the complex behavior and strategies of attackers due to insufficient information collection and dynamic change capabilities.

[0004] Therefore, the present invention urgently needs to provide a solution to improve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a power system honey array dynamic deployment defense method based on learning perception, which can solve the problem that the traditional honey array static deployment method has insufficient information collection ability and dynamic changes, and is difficult to effectively resist security attacks on the power system.

[0006] The present invention provides a method for dynamic deployment defense of power system honey array based on learning perception, comprising the following steps: S1. Deploy a honey array consisting of multiple honeypots in the power system; S2, obtain and aggregate local domain information of each honeypot based on message passing neural network; S3, combining the aggregated information with the current state information of the honeypot to extract the feature representation of the honeypot; S4, redirecting the attacker attacking the power system to the simulated power system when the feature indicates an abnormality; S5. updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model; S6. After updating the honeypot configuration, a set of algorithm sample data is obtained, the sample data is stored in the experience replay pool, and the parameters of the deep deterministic policy gradient algorithm model are updated based on the experience replay pool; S7. Repeat steps S2-S6 to continuously update the configuration of the honeypot to achieve dynamic deployment of the honey array.

[0007] The present invention provides a power system honey array dynamic deployment defense method based on learning perception, which improves the security threat perception capability by intelligently interacting with attackers based on a message passing neural network, and establishes a honey array dynamic deployment mechanism according to a deep deterministic policy gradient algorithm to enhance the deception of the simulation system, thereby improving the intelligence level and overall defense capability of the network defense system.

[0008] Optionally, a honey array consisting of multiple honeypots is deployed in the power system, wherein the honey array consists of nodes and edges, each honeypot in the honey array represents a node, and a path between two honeypots represents an edge.

[0009] Optionally, when acquiring and aggregating local neighborhood information of each honeypot based on a message passing neural network, the local neighborhood information includes a real-time attacker behavior data log file collected by the honeypot and traffic characteristics of the attacker on the attack path entering the honeypot.

[0010] Optionally, when acquiring and aggregating the local neighborhood information of each honeypot based on the message passing neural network, repeat the message acquisition phase times, during each iteration, through the message function To complete the update of local neighborhood information, the formula is defined as follows: , in, Is a node exist Round iteration information, is the update function, For the current node exist The hidden state information of Neighbor node exist Hidden state information, is the set of neighbor nodes.

[0011] Optionally, when acquiring and aggregating the local neighborhood information of each honeypot based on the message passing neural network, the aggregation function Aggregate the local neighborhood information obtained, and define the formula as follows: , in, Is a node exist Iterative aggregation of information.

[0012] Optionally, when the feature indicates an abnormality, an attacker attacking the power system is redirected to a simulated power system, wherein the simulated power system is composed of a special honeypot device simulating real system services, simulated management accounts, and a virtualized file system.

[0013] Optionally, when updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model, the state space of the value network model in the deep deterministic policy gradient algorithm model is defined as: , in, ; represents the state space, Indicates the current time State Space Information , , , Respectively represent the current time The intranet address, network port, and system service ID of the honeypot. Indicates the current time Behavioral attack pattern type, , , , Respectively represent the current time Network environment information such as throughput, round-trip latency, pulse signal strength, and attack frequency.

[0014] Optionally, when updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model, the action vector of the policy network model in the deep deterministic policy gradient algorithm model is defined as ,in, , represents the action vector matrix, Indicates changing the IP address of the honeypot. Indicates adjusting the open port. is the indicator function, when If it is 1, it means that the current system service ID is selected and the service is started, otherwise the current service is closed.

[0015] Optionally, when updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model, the reward in the deep deterministic policy gradient algorithm model is defined as, , in, represents the reward value for the attacker to be lured into the honeypot and interact at the current time t; It means that at the current time t the attacker continues to be active in the honeypot. , To continue the interaction time, To dynamically adjust the parameters, , is the number of invalid honeypots at the current time t, is the duration of the inactive honeypot, , , They are , , The sum of the three weight coefficients is 1.

[0016] Optionally, when updating the parameters of the deep deterministic policy gradient algorithm model based on the experience replay pool, the value network model parameters are updated using an algorithm that minimizes the mean square error, the policy network model parameters are updated based on the policy gradient method, and the network weights of the target network model are updated based on the soft update algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a logic diagram of a dynamic deployment defense method of honey array in power system based on learning perception. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0019] The present invention provides a power system honey array dynamic deployment defense method based on learning perception, such as Figure 1 As shown, the specific steps include: S1. Deploy a honey array consisting of multiple honeypots in the power system; S2, obtain and aggregate local domain information of each honeypot based on message passing neural network; S3, combining the aggregated information with the current state information of the honeypot to extract the feature representation of the honeypot; S4, redirecting the attacker attacking the power system to the simulated power system when the feature indicates an abnormality; S5. updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model; S6. After updating the honeypot configuration, a set of algorithm sample data is obtained, the sample data is stored in the experience replay pool, and the parameters of the deep deterministic policy gradient algorithm model are updated based on the experience replay pool; S7. Repeat steps S2-S6 to continuously update the configuration of the honeypot to achieve dynamic deployment of the honey array.

[0020] In fact, when executing step S1, honeypots are distributedly deployed on the front-end network access devices to form a honey array. The honeypots deploy a complete operating system and application programs, and set power system information as bait to attract attackers to record the attacker's command execution and file operations. The honey array simulates the real power system network environment, allowing attackers to perform comprehensive operations in it.

[0021] In fact, when executing step S2, when the attacker launches an attack on the honey bait, the MPNN model collects data from the honey array in real time through message passing, aggregation and update processes to achieve dynamic interaction with the attacker.

[0022] S21. Treat each honeypot as a node in the honey array , the behavior data of the nodes are collected as node information, and the traffic characteristics of the attacker on the honeypot attack path, such as throughput, round-trip delay, pulse signal strength and attack frequency, are defined as side information , used for side information collection.

[0023] MPNN uses an iterative information transmission algorithm to propagate information between nodes to enhance the system's detection and understanding of attack behaviors. In each iteration, the message function To complete the update of self-generated node information, , in, Is a node exist Round iteration information, Update function, For the current node exist The hidden state information of Neighbor node exist Hidden state information, is the set of neighbor nodes.

[0024] S22. Message aggregation refers to summarizing the messages of all neighboring nodes in order to generate the input information of the current node. The aggregation function is usually a weighted sum, average or maximum operation to ensure that the aggregation of messages does not depend on the order of neighboring nodes and maintain the symmetry of the message transmission process. Message aggregation uses a weighted method to generate the input information of the current node through the aggregation function. Aggregate the local neighborhood information obtained, and define the formula as follows:

[0025] in, Is a node exist Iterative aggregation of information.

[0026] In fact, when executing step S3, the aggregated information is combined with the state of the current node to extract the feature representation of the node. This update process is usually implemented through a nonlinear transformation such as a neural network layer, so that the node's features can gradually capture the information of its neighbors. The formula for message update can be expressed as, , in, For Node exist The characteristics of the wheel are shown in A learnable network.

[0027] In fact, when executing step S4, the intrusion attempt of the attacker who attacks the power system is directed to a special honeypot device, which simulates the simulation environment of the power system. By creating highly simulated system services, simulated management accounts, and virtualized file systems, the attacker will mistakenly believe that he has successfully invaded the target system. In this process, the real system is effectively isolated to avoid actual damage.

[0028] In fact, when executing step S5, it should be noted that the deep deterministic policy gradient algorithm model includes a policy network model and a value network model, each network has a corresponding target network, a total of four networks. The value network model is used to evaluate the expected value of each possible action in the current state, and the policy network model is used to output a set of continuous actions to update the configuration of the honey array based on the current state information, making it difficult for attackers to track the location information of the honeypot, and obtain rewards for measuring the performance of the output actions in the current state.

[0029] S51. Define the state space of the value network model as: , in, ; represents the state space, Indicates the current time State Space Information , , , Respectively represent the current time The intranet address, network port, and system service ID of the honeypot. Indicates the current time Behavioral attack pattern type, , , , Respectively represent the current time Network environment information such as throughput, round-trip latency, pulse signal strength, and attack frequency.

[0030] S52. Define the action vector of the policy network model as ,in, , represents the action vector matrix, Indicates changing the IP address of the honeypot. Indicates adjusting the open port. is the indicator function, when If it is 1, it means that the current system service ID is selected and the service is started, otherwise the current service is closed.

[0031] S53, in the deployment of honey arrays in power industrial control systems, the rewards are defined according to the defense objectives as follows: , One of the defense goals of the honeypot is to lure attackers into a fake environment, prevent attackers from directly contacting the real system, and collect attackers' behavior data through the honeypot. Therefore, when the attacker is successfully lured into the honeypot, the system should be given a positive reward. Here we assume that if the attacker is successfully deceived once, the reward value will be accumulated by 5. If the attacker successfully interacts with the system each time, the reward value will be accumulated. represents the reward value for the attacker to be lured into the honeypot and interact at the current time t; The second defense goal of the honey array is to lure attackers to continue activities in the honeypot, set increasing rewards according to the duration, and keep the honeypot attractive to attackers. It means that at the current time t the attacker continues to be active in the honeypot. , To continue the interaction time, To dynamically adjust parameters; The third defense goal of the honey array is to improve the efficiency of resource utilization. If the number of honeypots is too large or the configuration is redundant, it may lead to a waste of system resources. Therefore, if too many honeypots are deployed but not enough attackers are lured, negative rewards should also be given. When some honeypots have not been attacked or visited for a long time, negative rewards can be set to encourage the system to optimize deployment. Indicates the efficiency of resource utilization. , is the number of invalid honeypots at the current time t, is the duration of the inactive honeypot; , , They are , , The sum of the three weight coefficients is 1.

[0032] S54. Set up a policy network and target strategy network Weight and , set up the value network Target Value Network of and , set the experience replay buffer size (replaybuffer), discount factor and learning rate .

[0033] S55. At each time step, observe the current moment The state space , select the best action under the current network state, , Execute the current best action to update the honeypot configuration.

[0034] In fact, when executing step S6, the honeypot configuration is updated to obtain feedback from the environment, including the next state , Instant Rewards and whether it ends , this five-tuple Stored in the experience replay pool.

[0035] S61 randomly extracts a batch of samples from the experience replay pool. The default batch size is 256. Each sample consists of a five-tuple The random sampling of replay is conducive to the temporal correlation of samples and ensures the independence of data distribution, thereby improving the stability of learning.

[0036] S62, through the five-tuple Calculate the target Q value calculated by the target policy network and the target value network. The formula is as follows: , S63, in order to use the loss function that minimizes the mean squared error (MSE) to update the parameters of the value network , the formula is as follows, , S64, use the policy network to maximize the expected action value output by the value network, that is, the policy gradient method, and update the parameters through the policy network , the formula is as follows, , S65, where the gradient through the value network and the gradient of the policy network output Multiply to update the parameters, and finally use soft update to update the weights of the target network, making the update process of the target network smoother and more stable. The soft update formula is as follows: , in, is the soft update coefficient, which is usually set to a smaller value (such as ).

[0037] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. A power system honey array dynamic deployment defense method based on learning perception, characterized in that: The following steps are involved: S1. Deploy a honey array consisting of multiple honeypots in the power system; S2, obtain and aggregate local domain information of each honeypot based on message passing neural network; S3, combining the aggregated information with the current state information of the honeypot to extract the feature representation of the honeypot; S4, redirecting the attacker attacking the power system to the simulated power system when the feature indicates an abnormality; S5. updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model; S6. After updating the honeypot configuration, a set of algorithm sample data is obtained, the sample data is stored in the experience replay pool, and the parameters of the deep deterministic policy gradient algorithm model are updated based on the experience replay pool; S7. Repeat steps S2-S6 to continuously update the configuration of the honeypot to achieve dynamic deployment of the honey array.

2. According to the learning-based perception-based power system honey array dynamic deployment defense method of claim 1, it is characterized in that: A honey array consisting of multiple honeypots is deployed in the power system. The honey array consists of nodes and edges. Each honeypot in the honey array represents a node, and a path between two honeypots represents an edge.

3. According to the learning-based perception-based power system honey array dynamic deployment defense method of claim 1, it is characterized by: When acquiring and aggregating the local neighborhood information of each honeypot based on the message passing neural network, the local neighborhood information includes the real-time attacker behavior data log file collected by the honeypot and the traffic characteristics of the attacker on the attack path entering the honeypot.

4. According to the learning-aware power system honey array dynamic deployment defense method according to claim 1, it is characterized in that: When acquiring and aggregating the local neighborhood information of each honeypot based on the message passing neural network, the message acquisition phase is repeated times, during each iteration, through the message function To complete the update of local neighborhood information, the formula is defined as follows: ,in, Is a node exist Round iteration information, is the update function, For the current node exist The hidden state information of Neighbor node exist Hidden state information, is the set of neighbor nodes.

5. According to claim 4, a method for dynamic deployment defense of power system honey array based on learning perception is characterized in that: When acquiring and aggregating the local neighborhood information of each honeypot based on the message passing neural network, the aggregation function Aggregate the local neighborhood information obtained, and define the formula as follows: ,in, Is a node exist Iterative aggregation of information.

6. According to the learning-based perception-based power system honey array dynamic deployment defense method of claim 1, it is characterized by: When the characteristics indicate an abnormality, the attacker attacking the power system is redirected to a simulated power system, which is composed of a special honeypot device simulating real system services, simulated management accounts, and a virtualized file system.

7. The method for dynamic deployment of honey arrays in a power system based on learning perception according to claim 1 is characterized in that: When the honeypot configuration is updated based on the feature representation according to the deep deterministic policy gradient algorithm model, the state space of the value network model in the deep deterministic policy gradient algorithm model is defined as: ,in, ; represents the state space, Indicates the current time State Space Information , , , Respectively represent the current time The intranet address, network port, and system service ID of the honeypot. Indicates the current time Behavioral attack pattern type, , , , Respectively represent the current time Network environment information such as throughput, round-trip latency, pulse signal strength, and attack frequency.

8. The method for dynamic deployment of honey arrays in a power system based on learning perception according to claim 1 is characterized in that: When updating the honeypot configuration based on the feature representation according to the deep deterministic policy gradient algorithm model, the action vector of the policy network model in the deep deterministic policy gradient algorithm model is defined as ,in, , represents the action vector matrix, Indicates changing the IP address of the honeypot. Indicates adjusting the open port. is the indicator function, when If it is 1, it means that the current system service ID is selected and the service is started, otherwise the current service is closed.

9. The method for dynamic deployment of honey arrays in a power system based on learning perception according to claim 1 is characterized in that: When the honeypot configuration is updated based on the feature representation according to the deep deterministic policy gradient algorithm model, the reward in the deep deterministic policy gradient algorithm model is defined as, ,in, Indicates the current time The reward value for an attacker to be lured into the honeypot and interact with it; Indicates the current time The attacker continues to operate in the honeypot. , For continuous interaction time, To dynamically adjust the parameters, , For the current moment The number of invalid honeypots, is the duration of the inactive honeypot, , , They are , , The sum of the three weight coefficients is 1.

10. The method for dynamic deployment of honey arrays in power systems based on learning perception according to claim 1 is characterized in that: When updating the parameters of the deep deterministic policy gradient algorithm model based on the experience replay pool, the value network model parameters are updated using an algorithm that minimizes the mean square error, the policy network model parameters are updated based on the policy gradient method, and the network weights of the target network model are updated based on the soft update algorithm.

Images