A network system defense method based on control and guard graph

Abstract

The present invention provides a network system defense method based on a control and defense graph, which relates to the field of network security technology. The method comprises: obtaining honeypot device information, network asset device information, and attack path information to generate a control and defense graph with multiple paths; simulating the network asset devices most vulnerable to attack on the paths based on the control and defense graph to generate corresponding honeypot devices, and obtaining attacker attack intent information based on the honeypot devices; inputting the attack intent information into a defense benefit model for training and parameter adjustment to obtain a trained defense benefit model; outputting protection strategy information that maximizes the defender's benefits based on the trained defense benefit model; and the defender performing defense by replacing the IP address or port number of the honeypot device based on the protection strategy information. In the context of the control and defense graph, the present invention utilizes a reinforcement learning algorithm to dynamically adjust the defense strategy, thereby increasing the attack cost for attackers and improving the security of the network system.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a network system defense method based on a control graph. Background Art

[0002] In recent years, with the rapid development and widespread adoption of network technology, the complexity and importance of network information systems have increased. Today, cyberspace is not only a focus of attention for governments, industry, and academia, but is also closely intertwined with everyone's daily lives. However, with this progress, the security threats facing cyberspace are also becoming increasingly significant. In particular, cyberattacks are becoming increasingly diverse, covert, and intelligent, making existing defense systems often difficult to protect against. Therefore, the research on an effective network security defense decision-making system is of great significance.

[0003] Prior art proposes an innovative adversarial model for the adversarial relationship between attackers and defenders in computer systems. A key feature of this model is that it accounts for the perception asymmetry between attackers and defenders, namely, the differences in their understanding and utilization of environmental information. To effectively simulate attack scenarios, prior art constructs basic attack scenarios based on the key stages of the cyberattack kill chain. This approach not only helps understand the different stages of cyberattacks but also provides a practical framework for analyzing attacker behavior patterns. Given the inherent complexity of the model and the large number of potential states, determining the optimal defense strategy through traditional analytical methods is extremely difficult. To address this issue, prior art employs the DQN algorithm to train a defense agent. This deep reinforcement learning approach enables the defense agent to intelligently select the most effective defense strategy based on observed attacker behavior. Overall, prior art, by introducing deep reinforcement learning, offers a new perspective in the field of cybersecurity, particularly in scenarios involving complex interactions between attackers and defenders. It not only enhances our understanding of cyberattack dynamics but also provides technical support for the development of efficient and adaptable defense mechanisms.

[0004] However, in the actual network information system environment, defenders are faced with the task of comprehensive defense of the entire network system. This requires not only targeting a single node, but multiple nodes, and requires in-depth analysis and protection of the entire network topology. Existing technical solutions mainly focus on the optimal defense strategy for a single host node facing different attack stages, but do not fully consider the defender's coherent response in a complex network topology, resulting in low defense effectiveness.

[0005] Therefore, it is urgent to provide a solution to improve the above problems. Summary of the Invention

[0006] To improve the above problems, the present invention provides a network system defense method based on a control graph to improve the current problem of low defense effectiveness of the defender in network information systems.

[0007] The present invention provides a network system defense method based on a control graph, which adopts the following technical solutions:

[0008] Obtaining honeypot device information, network asset device information, and attacker's attack path information to generate a control-guard graph with multiple paths;

[0009] Obtain attack value scores of all M network asset devices on the path based on the protection vulnerability, sort the attack value scores from high to low, select the honeypot devices corresponding to the first N network asset devices as high attack value honeypot devices, and obtain the attacker's attack intention information based on the high attack value honeypot devices, where N < M;

[0010] A defense benefit model is trained based on the attack intention information to obtain a trained defense benefit model, a defense strategy that maximizes the defender's benefit is obtained based on the trained defense benefit model, and defense adjustments are made to the high attack value honey spots according to the defense strategy; the defense strategy includes one of IP adjustment and port number adjustment.

[0011] The beneficial effects of the network system defense method based on the control and defense graph provided by the present invention are: first, the method can intuitively display the attacker's attack intention through the control and defense graph, broaden the global perspective of the network defense party, enhance the prediction and response capabilities of potential threats, and ensure that the defense measures can be more accurate and effective when facing complex and changeable network attacks; second, according to the attack intention information, the IP address and port number of the honeypot device are changed using the defense benefit model, which not only increases the attacker's attack cost, but also continuously monitors network activities, timely captures and analyzes the attacker's suspicious behavior, and improves the intelligence level of the network defense system.

[0012] Optionally, the higher the protection vulnerability of the network asset device, the higher the attack value of the attacker.

[0013] Optionally, the process of training the defense benefit model includes:

[0014] Establishing a state space and an action space, where the state space consists of the current configuration of the network and the security level of network assets, and the action space consists of changing the IP address and port number of the honeypot device;

[0015] Initialize the time step and the weights of the main network and the target network, and initialize the network parameters of the target value to the main network parameters;

[0016] The state space is input into the main network, and the corresponding action that maximizes the action-value function value is selected as the optimal action at the current moment. The optimal action is to change one of the IP address and port number of the honeypot device.

[0017] The defender executes the optimal action with a preset probability based on the current state, and randomly selects an action from the action space to execute, corresponding to the reward and the state at the next moment;

[0018] Based on the current state, the state at the next moment, and the reward, the transition data corresponding to each time step is obtained, and the transition data is stored in the playback buffer, where the transition data consists of multiple transition data samples.

[0019] A transfer data sample is obtained by randomly sampling from the playback buffer, and a target action-value function value and a current action-value function value are calculated based on the transfer data sample;

[0020] A loss function is calculated based on the difference between the target action-value function value and the current action-value function value, and the weights of the main network and the target network are updated using gradient descent based on the loss function.

[0021] Optionally, the attack intention information consists of attacks against IP addresses and attacks against port numbers, wherein attacks against specific IP addresses include: distributed denial of service attacks, IP scanning attacks, targeted attacks, and attacks against ports include: port scanning attacks, service-specific attacks.

[0022] Optionally, the reward is:

[0023] R t =αr t -βc t +γ(ω IP ·a IP +ω Port ·a Port );

[0024] Among them, R t represents the reward obtained by the defender, α, β, γ are weight coefficients used to balance the influence of each part, r t The attacker's feedback is 1 for success and 0 for failure. t is the resource overhead of the honeypot device after the IP address or port mutation, a IP Indicates IP address mutation, a Port Indicates port mutation, ω IP is the weight of the IP address mutation strategy, ω Port is the weight of the port mutation strategy.

[0025] Optionally, the defense strategy includes:

[0026] When an attacker attacks the specific IP address, the weight of the IP address mutation strategy is increased; when an attacker attacks the port vulnerability, the weight of the port mutation strategy is increased, wherein the sum of the weight of the port mutation strategy and the weight of the IP address mutation strategy is 1.

[0027] Optionally, the process of adjusting the defense of the high attack value sweet spot according to the defense strategy includes:

[0028] Based on the defense strategy, a preset IP address and a preset port are randomly selected from an unassigned IP address pool and a port pool, and the preset IP address and the preset port are iteratively updated based on a fixed mutation cycle, a weight of the IP address mutation strategy, and a weight of the port mutation strategy to obtain an updated IP address and an updated port, and after each iterative update, the preset IP address and the preset port before the iterative update are removed from the IP address pool and the port list pool. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of a network system defense method based on a control graph provided by the present invention is shown;

[0030] Figure 2 It represents the attacker intention identification flow chart provided by the present invention;

[0031] Figure 3 The figure shows the overall framework of a network system defense method based on a control graph provided by the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0033] See Figure 1 The embodiment of the present invention provides a network system defense method based on a control graph, comprising the following steps:

[0034] S1. Obtain the honeypot device information, network asset device information, and attacker's attack path information to generate a control-guard graph with multiple paths;

[0035] S2. Obtain attack value scores of all M network asset devices on the path based on the protection vulnerability, sort the attack value scores from high to low, select the honeypot devices corresponding to the first N network asset devices as high attack value honeypot devices, and obtain the attacker's attack intention information based on the high attack value honeypot devices, where N < M;

[0036] S3. Training a defense benefit model based on the attack intent information to obtain a trained defense benefit model, obtaining a defense strategy that maximizes the defender's benefit based on the trained defense benefit model, and performing defense adjustments on the high attack value honey spot according to the defense strategy; the defense strategy includes one of IP adjustment and port number adjustment.

[0037] In order to describe the present invention more clearly, first, the following terms are explained:

[0038] An attack graph is a visual representation of potential attack paths within a network or system. This graphical representation helps network security experts understand and identify security vulnerabilities within the network.

[0039] Control and defense graph: The control and defense graph is based on the attack graph by adding multiple network asset devices and multiple honeypot devices. Network asset devices include FTP servers, hosts, and databases. Honeypot devices include file honeypot devices, host honeypot devices, and account honeypot devices.

[0040] A honeypot is a virtual decoy device that interacts directly with attackers, based on a traditional honeypot. It simulates network assets and has the same appearance, performance, and network behavior as these assets. Implemented using container technology, it boasts lightweight, easy orchestration, and fast deployment, making it easily adaptable to both local networks and cloud scenarios. The honeypot system in this invention can flexibly and seamlessly integrate into real network environments, creating a hybrid environment of fake and real for attackers.

[0041] In some embodiments, see Figure 2 , which represents a simple example of a control and defense graph. The intrusion detection system (IDS), firewall, switch and Web server are network asset devices. Among them, the Web server is composed of FTP server, host, database, etc. After simulating the FTP server, host and database, the corresponding file honey points, host honey points and account honey points are obtained.

[0042] In some embodiments, see Figure 3The honey spot log includes file honey spots, traffic honey spots, host honey spots and Web simulation honey spots. The attacker's intention information is updated based on the security device log and the honey spot log to obtain the updated attack intention information. The updated attack intention information interacts with the honey array controller to update the control and defense graph, and continuously trains and adjusts parameters through the defense benefit model to achieve dynamic tuning of the honey spot strategy. Among them, the honey spot strategy consists of changing the IP address or port number of the honey spot device.

[0043] In some embodiments, when executing step S1, the higher the protection vulnerability of the network asset device, the higher the attack value of the attacker.

[0044] In fact, the higher the protection vulnerability of network asset equipment and the weaker its defense capability, the higher the attack value of such network asset equipment for attackers.

[0045] In some embodiments, when executing step S2, the attack intention information consists of attacks against IP addresses and attacks against port numbers, among which attacks against specific IP addresses are: distributed denial of service attacks, IP scanning attacks, targeted attacks, and attacks against ports are: port scanning attacks, service-specific attacks.

[0046] Specifically, a distributed denial of service attack refers to an attacker concentrating all attack resources to attack a single IP address, and a targeted attack refers to an attacker launching an attack on high-value asset information within the control graph.

[0047] In some embodiments, when executing step S3, the process of training the defense benefit model includes:

[0048] S3 1. Establish a state space and an action space. The state space consists of the current configuration of the network and the security level of the network assets. The action space consists of changing the IP address and port number of the honeypot device.

[0049] S32, initialize the time step and the weights of the main network and the target network, and initialize the network parameters of the target value to the main network parameters;

[0050] S33: Input the state space into the main network, and select the action that maximizes the action-value function as the optimal action at the current moment, wherein the optimal action is to change one of the IP address and port number of the honeypot device;

[0051] S34. The defender executes the optimal action with a preset probability based on the current state, and randomly selects an action from the action space to execute, corresponding to the reward and the state at the next moment;

[0052] S35. Acquire transfer data corresponding to each time step based on the current state, the state at the next moment, and the reward, and store the transfer data in a playback buffer, wherein the transfer data consists of multiple transfer data samples;

[0053] S36, obtaining a transfer data sample by random sampling from the playback buffer, and calculating a target action-value function value and a current action-value function value based on the transfer data sample;

[0054] S37. Calculate the loss function based on the difference between the target action-value function value and the current action-value function value, and update the weights of the main network and the target network using gradient descent based on the loss function.

[0055] Specifically, when executing step S31, the state space X t It can be expressed as: X t ={X1, X2}, where X1 represents the current configuration of the network and X2 represents the security level of the network asset equipment.

[0056] Furthermore, when executing step S32, the time step is initialized to t=1, 2, 3...M, where M represents the maximum number of iterations; the weight β of the main network is initialized, and the weight β′ of the target network is initialized to β.

[0057] Furthermore, when executing step S33, the process of selecting the corresponding action that maximizes the action-value function value as the optimal action at the current moment includes:

[0058] X t Input into the main network and select the action that maximizes the action-value function As the best action at the current moment, Among them, Q(X t ,α,β) represents the attacker’s action according to state X under weight β t The action-value function value obtained by performing action α.

[0059] Furthermore, when executing step S34, the defender executes the optimal action with a preset probability based on the current state and randomly selects an action from the action space to execute. The attack action executed by the attacker consists of two parts: the random action and the optimal action, specifically including:

[0060] The attacker performs the best action with a preset probability P1 Randomly select an attack action from the action set with a probability of 1-P1 as the attacker's attack action Get the attacker's attack action a at the current moment t , which is expressed as:

[0061]

[0062] In fact, when executing step S34, the reward is:

[0063] R t =αr t -βc t +I(ω IP ·a IP +ω Port ·a Port );

[0064] Among them, R t represents the reward obtained by the defender, α, β, and I are weight coefficients used to balance the influence of each part, and r t The attacker's feedback is 1 for success and 0 for failure. t is the resource overhead of the honeypot device after the IP address or port mutation, a IP Indicates IP address mutation, a Port Indicates port mutation, ω IP is the weight of the IP address mutation strategy, ω Port is the weight of the port mutation strategy.

[0065] In fact, when executing step S35 to obtain the transfer data corresponding to each time step, the transfer data is expressed as: (X t , a t , R t , X t+1 ), where X t+1 is the state space at the next moment.

[0066] Furthermore, when executing step S36, the process of calculating the target action-value function value and the current action-value function value based on the transfer data sample includes:

[0067] For each transfer data sample, calculate the target action-value function value: y i =r i +γQ(X t+1 , argmaxQ(X t+1 , a t+1 ; β); β′), calculate the action-value function value Q(X t , a t ; β), where γ represents the discount factor, a t+1 Indicates the action at the next moment, a t Indicates the action at the current moment.

[0068] Furthermore, when executing step S37, the loss function LOSS(β) is expressed as the square of the difference between the target action-value function value and the action-value function value, expressed as:

[0069] LOSS(β)=(y i -Q((X t , a t ;β)) 2 ;

[0070] Furthermore, when executing step S3, the defense strategy includes:

[0071] When the attacker attacks the specific IP address, the weight of the IP address mutation strategy is increased. IP When the attacker attacks the port vulnerability, the weight of the port mutation strategy is increased ω Port , where the sum of the weight of the port mutation strategy and the weight of the IP address mutation strategy is 1, that is, ω Port +ω IP =1.

[0072] Specifically, at the initial moment, the weight of the port mutation strategy and the weight of the IP address mutation strategy are both 0.5. When the attacker attacks the port vulnerability, the weight of the port mutation strategy increases to 0.9 and the weight of the IP address mutation strategy decreases to 0.1. When the attacker attacks the IP address, the weight of the IP address mutation strategy increases to 0.9 and the weight of the port mutation strategy decreases to 0.1.

[0073] In fact, by increasing the weight of the IP address mutation strategy or the port mutation strategy, the attacker's original attack target will be invalidated, forcing the attacker to change the attack strategy, that is, to choose back and forth between the port mutation strategy and the IP address mutation strategy, which prolongs the attacker's attack cycle and increases the attacker's attack cost.

[0074] In some embodiments, the process of performing defense adjustment on the high attack value sweet spot in step S3 includes:

[0075] Based on the defense strategy, a preset IP address and a preset port are randomly selected from an unassigned IP address pool and a port pool, and the preset IP address and the preset port are iteratively updated based on a fixed mutation cycle, a weight of the IP address mutation strategy, and a weight of the port mutation strategy to obtain an updated IP address and an updated port, and after each iterative update, the preset IP address and the preset port before the iterative update are removed from the IP address pool and the port list pool.

[0076] In fact, the process of iteratively updating the preset IP address and the preset port is implemented by using the forwarding control module of the SDN, and the IP address and the port number are periodically updated according to the mutation cycle.

[0077] While the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations of these embodiments are possible. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the claims. Furthermore, the invention described herein is susceptible to other embodiments and may be practiced or implemented in a variety of ways.

Claims

1. A network system defense method based on a control graph, characterized in that: include: Obtaining honeypot device information, network asset device information, and attacker's attack path information to generate a control-guard graph with multiple paths; Obtain attack value scores of all M network asset devices on the path based on the protection vulnerability, sort the attack value scores from high to low, select the honeypot devices corresponding to the first N network asset devices as high attack value honeypot devices, and obtain the attacker's attack intention information based on the high attack value honeypot devices, where N < M; A defense benefit model is trained based on the attack intention information to obtain a trained defense benefit model, a defense strategy that maximizes the defender's benefit is obtained based on the trained defense benefit model, and defense adjustments are made to the high attack value honey spot device according to the defense strategy; the defense strategy includes one of IP adjustment and port number adjustment.

2. A network system defense method based on a control graph according to claim 1, characterized in that: The higher the protection vulnerability of network asset equipment, the higher the attack value of the attacker.

3. The network system defense method based on the control graph according to claim 1 is characterized in that: The process of training the defense benefit model includes: Establishing a state space and an action space, where the state space consists of the current configuration of the network and the security level of network assets, and the action space consists of changing the IP address and port number of the honeypot device; Initialize the time step and the weights of the main network and the target network, and initialize the network parameters of the target value to the main network parameters; Input the state space into the main network and select the action that maximizes the action-value function as the optimal action at the current moment. The optimal action is to change one of the IP address and port number of the honeypot device. The defender executes the optimal action with a preset probability based on the current state, and randomly selects an action from the action space to execute, corresponding to the reward and the state at the next moment; Acquire transfer data corresponding to each time step based on the current state, the state at the next moment, and the reward, and store the transfer data in a replay buffer, wherein the transfer data consists of multiple transfer data samples; A transfer data sample is obtained by randomly sampling from the playback buffer, and a target action-value function value and a current action-value function value are calculated based on the transfer data sample; A loss function is calculated based on the difference between the target action-value function value and the current action-value function value, and the weights of the main network and the target network are updated using gradient descent based on the loss function.

4. The network system defense method based on the control graph according to claim 1 is characterized in that: The attack intention information consists of attacks against IP addresses and attacks against port numbers. Among them, attacks against specific IP addresses include: distributed denial of service attacks, IP scanning attacks, and targeted attacks; attacks against ports include: port scanning attacks and service-specific attacks.

5. The network system defense method based on the control graph according to claim 3 is characterized in that: The rewards are: ； in, represents the reward obtained by the defender, , is the weight coefficient, which is used to balance the influence of each part. The feedback of whether the attacker succeeded or not, 1 for success and 0 for failure, is the resource overhead of the honeypot device after the IP address or port number changes. Indicates a mutation in the IP address. Indicates a port mutation. is the weight of the IP address mutation strategy, is the weight of the port mutation strategy.

6. The network system defense method based on the control graph according to claim 1 is characterized in that: The defense strategies include: When an attacker attacks a specific IP address, the weight of the IP address mutation strategy is increased; when an attacker attacks a port vulnerability, the weight of the port mutation strategy is increased, wherein the sum of the weights of the port mutation strategy and the IP address mutation strategy is 1.

7. The network system defense method based on control graph according to claim 1, characterized in that: The process of adjusting the defense of the high attack value honeypot device according to the defense strategy includes: Based on the defense strategy, a preset IP address and a preset port are randomly selected from an unassigned IP address pool and a port pool, and the preset IP address and the preset port are iteratively updated based on a fixed mutation cycle, a weight of the IP address mutation strategy, and a weight of the port mutation strategy to obtain an updated IP address and an updated port, and after each iterative update, the preset IP address and the preset port before the iterative update are removed from the IP address pool and the port list pool.

Images