Network defense method, control device and storage medium

Abstract

The present application provides a network defense method, a control device and a storage medium. After the control device establishes an attack and defense game model, it repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior type of the identity type of the attack terminal. After the operation, the optimal defense strategy and the optimal attack strategy for the target working terminal are determined according to the attack strategy information and the prior probability, and the optimal defense strategy is executed. The control device calculates the Markov learning function value according to the optimal defense strategy and the attack strategy information until it stops the loop when it is determined that the target working terminal is not in a safe state according to the Markov learning function value. The control device determines the attack state of the target working terminal according to the continuous attack strategy information obtained by the target working terminal, which helps it determine the covert attack according to the continuously obtained information, so that it can better protect the target working terminal.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of network security technology, and in particular, to a network defense method, a control device, and a storage medium. Background Art

[0002] As we enter the era of the Internet of Everything, the security issues of various terminal devices are becoming increasingly severe. The existing network defense architecture based on passive scanning intrusion detection is not enough for current network attacks. The existing method is that the security network device extracts the effective relationship path from the interactive relationship between the items to be detected in the data to be detected (for example: IP address, interface call, program execution, etc.), and performs anomaly detection on the effective relationship path to obtain the threat detection result of the target host, wherein the effective relationship path includes at least two detection items and the interactive relationship between the detection items. However, it is difficult for the network security device to identify unknown types of network attacks that are highly concealed, large-scale and intelligent. Summary of the invention

[0003] The present application provides a network defense method, a control device and a storage medium, which are used to solve the technical problem that hidden and continuous attack operations of security network devices are difficult to detect and timely defense cannot be achieved.

[0004] In a first aspect, the present application provides a network defense method, the method is applied to a control device, the control device is located in a target system, the target system includes at least one working terminal and a honeynet cluster, the method includes:

[0005] Establishing an attack-defense game model; wherein the attack-defense game model includes a plurality of game participants, a strategy space, a signal space, a priori probability, a posterior probability and the benefits of each of the game participants;

[0006] Repeatedly obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior probability of the identity type of the attack terminal;

[0007] Determine, according to the attack strategy information and the prior probability, an optimal defense strategy and an optimal attack strategy of the attack terminal for the optimal defense strategy, and execute the optimal defense strategy;

[0008] Calculating a Markov learning function value according to the optimal defense strategy and the attack strategy information;

[0009] According to the Markov learning function value, it is determined whether the target working terminal is in a safe state and the loop is terminated when the target working terminal is not in a safe state.

[0010] In the above technical scheme, after the control device establishes the attack and defense game model, it obtains the prior probability and the attack strategy information observed by the control device in at least one cycle, and determines the optimal defense strategy of the control device against the attack terminal and the optimal attack strategy of the attack terminal for the optimal defense strategy according to the prior probability and the attack strategy information in each cycle, and uses the Markov learning function to determine whether the target working terminal is dangerous under the influence of this attack strategy, so that the control device terminates the cycle when it determines that the target working terminal is not in a safe state, so that the control device monitors multiple attacks performed by the attack terminal, and counts the damage caused by multiple attacks to the target system, and perceives the hidden and continuous attack operations, so that the control device can timely protect the target working terminal when it determines that the target working terminal is not in a safe state, and effectively ensure the safety of the target working terminal.

[0011] Optionally, determining an optimal defense strategy and an optimal attack strategy for the target working terminal according to the attack strategy information and the prior probability specifically includes:

[0012] Calculating the posterior probability of the identity type of the attacking terminal according to the attack strategy information and the prior probability;

[0013] Determining the identity type of the attacking terminal according to the posterior probability;

[0014] Determining system cost and type cost according to the attack strategy information and the identity type;

[0015] Calculating the attacker benefit of the attack terminal and the defender benefit of the target system according to the system cost, the type cost, the attack strategy information and the posterior probability;

[0016] An optimal defense strategy and an optimal attack strategy for the target working terminal are determined according to the posterior probability and the defender benefit.

[0017] Optionally, determining a system cost and a type cost according to the attack strategy information and the identity type specifically includes:

[0018] Searching the attack strategy information in a system cost mapping table to obtain a system cost corresponding to the attack strategy information;

[0019] The attack strategy information and the identity type are searched in a type cost mapping table to obtain a type cost corresponding to the attack strategy information.

[0020] Optionally, the strategy space of the game participants includes a defense strategy space, and the defense strategy space includes at least one defense strategy; and calculating the Markov learning function value according to the optimal defense strategy and the attack strategy information specifically includes:

[0021] Calculate the attack and defense reward value of the target working terminal in the current state according to the optimal defense strategy and the attack strategy information;

[0022] Obtaining the cumulative reward value of the target work terminal from the initial state to the current state and the Markov learning function value of the target work terminal in the current state;

[0023] Calculate the maximum cumulative reward value of the target working terminal in the next state according to the optimal attack strategy of the attack terminal for the optimal defense strategy, all the defense strategies, the cumulative reward value and the attack and defense reward value;

[0024] The Markov learning function value of the target working terminal in the next state of the current state is calculated according to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state and the Markov learning function value in the current state.

[0025] Optionally, the strategy space of the game participant further includes an attack strategy space, and the attack strategy space includes at least one attack strategy; and calculating the attack and defense reward value of the target working terminal in the current state according to the optimal defense strategy and the attack strategy information, specifically includes:

[0026] According to the optimal defense strategy, the attack strategy information and the attack and defense reward value calculation formula, the attack and defense reward value of the target working terminal in the current state is calculated:

[0027] The attack and defense reward value calculation formula specifically includes:

[0028]

[0029] Among them, s p represents the pth state after the current state of the target working terminal, Re(f x (s p)) represents the attack and defense reward value of the target work terminal x in state p, i represents the defense identifier in the defense strategy space, j represents the attack identifier in the attack strategy space, i=0 represents that the target work terminal x is not defended in state p, i≠0 represents that the target work terminal x is defended by the i-th defense strategy in state p, j=0 represents that the target work terminal x is not attacked in state p, j≠0 represents that the target work terminal x is attacked by the j-th attack strategy in state p, P represents the protection value, D i represents the reward value when the target working terminal x is protected by the i-th defense strategy, δ i represents the reward coefficient, D represents the attack reward value when the target working terminal is attacked without being protected by the defense strategy, and Pk ij It represents the reward value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

[0030] Optionally, calculating the maximum cumulative reward value of the target work terminal in the next state according to the optimal attack strategy of the target work terminal, all the defense strategies, the cumulative reward value and the attack and defense reward value specifically includes:

[0031] According to the optimal attack strategy of the attack terminal for the optimal defense strategy, each of the defense strategies and the attack and defense reward value formula, calculate the attack and defense reward value of the target working terminal in the next state adjusted based on each of the defense strategies;

[0032] Adding each of the attack and defense reward values ​​to the cumulative reward value, and calculating the cumulative reward value of the target working terminal in the next state adjusted based on each of the defense strategies;

[0033] The maximum value of the accumulated reward values ​​in the next state corresponding to each of the defense strategies is determined as the maximum accumulated reward value.

[0034] Optionally, calculating the Markov learning function value of the target working terminal in the next state of the current state according to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state, and the Markov learning function value in the current state specifically includes:

[0035] According to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state, the Markov learning function value in the current state and the learning function update formula, the Markov learning function value of the target working terminal in the next state of the current state is calculated; wherein the learning function update formula specifically includes:

[0036]

[0037]

[0038] in, represents the Markov learning function value of the target working terminal in the next state of the current state, represents the Markov learning function value of the target working terminal in the current state, α represents the learning rate parameter, γ represents the discount rate parameter, represents the maximum cumulative reward value in the next state; s0 represents the initial state of the target work terminal, Re(f x (s0)) represents the attack and defense reward value corresponding to the initial state; s1 represents the next state of the target working terminal state s0, Re(f x (s1)) represents the attack and defense reward value corresponding to state s1; s p Represents the current state of the terminal, Re(f x (s p )) represents the attack and defense reward value corresponding to the current state; s p+1 represents the next state of the current state of the target working terminal, Re(f x (s p+1 )) represents the attack and defense reward value corresponding to the next state.

[0039] Optionally, determining whether the target working terminal is at a safe level according to the Markov learning function value specifically includes:

[0040] When the Markov learning function value is less than or equal to the safety threshold, determining that the target working terminal is in the safety state;

[0041] When the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in the safety state.

[0042] In a second aspect, the present application provides a control device, including: a processor and a memory communicatively connected to the processor;

[0043] Memory stores computer instructions;

[0044] When executing computer instructions, the processor is used to implement the network defense method involved in the first aspect.

[0045] In a third aspect, the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the network defense method involved in the first aspect.

[0046] The present application provides a network defense method, a control device and a storage medium. The control device establishes an attack and defense game model, and repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior type of the identity type of the attack terminal. After the operation, the optimal defense strategy and the optimal attack strategy for the target working terminal are determined according to the attack strategy information and the prior probability, and the optimal defense strategy is executed. The control device calculates the Markov learning function value according to the optimal defense strategy and the attack strategy information until it is determined according to the Markov learning function value that the target working terminal is not in a safe state, and stops the loop, so that the control device determines the attack state of the target working terminal according to the continuous attack strategy information obtained by the target working terminal, which helps it to determine the covert attack according to the continuously obtained information, and improves its better protection of the target working terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] Figure 1 This is an application scenario diagram of a network defense method provided by the present application according to an exemplary embodiment;

[0049] Figure 2 A flowchart of a network defense method provided by the present application according to an exemplary embodiment;

[0050] Figure 3 A schematic diagram of a calculation flow of a Markov learning function value provided by the present application according to an exemplary embodiment;

[0051] Figure 4 A schematic diagram of the structure of a network defense device provided by the present application according to an embodiment;

[0052] Figure 5 This is a schematic diagram of the structure of a control device provided according to an embodiment of the present application.

[0053] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0054] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0055] As we enter the era of the Internet of Everything, the security issues of various terminal devices are becoming increasingly severe. The existing network defense architecture based on passive scanning intrusion detection is not enough for current network attacks. The existing method is that the security network device extracts the effective relationship path from the interactive relationship between the items to be detected in the data to be detected (for example: IP address, interface call, program execution, etc.), and performs anomaly detection on the effective relationship path to obtain the threat detection result of the target host, wherein the effective relationship path includes at least two detection items and the interactive relationship between the detection items. However, it is difficult for the network security device to identify unknown types of network attacks that are highly concealed, large-scale and intelligent.

[0056] The present application provides a network defense method, a control device, and a storage medium, which are intended to solve the technical problem that the security network device is not easy to detect and cannot achieve timely defense against concealed and continuous attack operations. The technical concept of the present application is: the control device establishes an attack and defense game model, repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior type of the identity type of the attack terminal, after the operation, according to the attack strategy information and the prior probability, determine the optimal defense strategy and the optimal attack strategy for the target working terminal, and execute the optimal defense strategy, the control device calculates the Markov learning function value according to the optimal defense strategy and the attack strategy information, until it is determined that the target working terminal is not in a safe state according to the Markov learning function value, stops the loop, forwards the request of the working terminal to protect the target working terminal from being attacked again, so that the control device determines the attack state of the target working terminal according to the continuous attack strategy information obtained by the target working terminal, which helps it determine the concealed attack according to the continuously obtained information, so that it can better protect the target working terminal.

[0057] Figure 1 This is an application scenario diagram of a network defense method provided by this application according to an exemplary embodiment, such as Figure 1As shown, the application scenario diagram includes: an attack terminal 106, a routing device 105, a first OpenFlow switch 102, a second OpenFlow switch 103, a third OpenFlow switch 104, a control device 101, an intrusion detection device 108, an SDN controller 107, a workspace 109, and a honeynet cluster 111. The workspace 109 includes a plurality of work terminals, and the honeynet cluster 111 includes a plurality of honeynet virtual terminals, in which a system for virtual mapping of the work terminals in the workspace 109 is installed. The attack terminal 106 is connected to the routing device 105, the routing device 105 is connected to the first OpenFlow switch 102, the first OpenFlow switch 102 is respectively connected to the second OpenFlow switch 103, the third OpenFlow switch 104, the control device 101, and the SDN controller 107, the intrusion detection device 108 is connected to the SDN controller 107, the workspace 109 is connected to the second OpenFlow switch 103, and the honeynet cluster 111 is connected to the third OpenFlow switch 104.

[0058] When the attack terminal 106 sends an access request to the target work terminal 110 in the work area 109, the attack terminal 106 sends the access request to the routing device 105, and the routing device 105 forwards the access request to the first OpenFlow switch 102. The first OpenFlow switch 102 forwards the access request to the target work terminal 110 through the second OpenFlow switch 103. At the same time, the first OpenFlow switch 102 sends the access request to the intrusion detection device 108 through the SDN controller 107. The intrusion detection device 108 is installed with an intrusion detection system (IDS for short). The IDS identifies the above access request and detects whether it has the risk of malicious attack on the target work terminal 110.

[0059] If there is a risk, the detection result is fed back to the control device 101, and the control device 101 controls the first OpenFlow switch 102 to forward the access request sent again by the attack terminal 106 to the target honeynet device 112 in the honeynet cluster 111, which is loaded with the image of the target work terminal 110. The target honeynet device 112 is hidden behind the firewall, and all incoming and outgoing data are monitored, captured and controlled; if there is no risk, and the target work terminal 110 determines that the access request belongs to the abnormal operation type, the target work terminal 110 feeds back the determined result to the control device 101, and the control device 101 determines the corresponding defense strategy through the attack and defense game model, and determines whether the target work terminal 110 is safe when receiving the next access request according to the Markov learning function. More specifically, the control device 101 establishes an attack and defense game model, and obtains the attack strategy information in the access request observed by the control device 101, the prior probability of the identity type of the attack terminal 106, the system cost and the type cost, wherein the identity type of the attack terminal 106 includes a user (User) or an attacker (Attacker). The control device 101 determines the optimal defense strategy and the optimal attack strategy of the attack terminal 106 for the optimal defense strategy according to the attack strategy information, prior probability, system cost and type cost, and executes the optimal defense strategy. At the same time, according to the optimal defense strategy and attack strategy information, the value of the Markov learning function of the target work terminal 110 in the current state is calculated to determine whether the target work terminal 110 is in a safe state. When the target work terminal 110 is in a safe state, when the target work terminal 110 obtains an access request again, the control device 101 still determines the defense strategy to protect the target work terminal 110 through the attack and defense game model; when the target work terminal 110 is not in a safe state, when the target work terminal 110 obtains an access request again, the control device 101 directly forwards the access request to the honeynet cluster 111 to protect the target work terminal 110 from being attacked by the attack terminal 106 again.

[0060] Based on the above application scenarios, some implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the timing of the steps in the following method embodiments is only an example and not a strict limitation.

[0061] Figure 2 This is a flow chart of a network defense method provided by the present application according to an exemplary embodiment, such as Figure 2 As shown, network defense methods include:

[0062] S201. Control the device to establish an attack and defense game model.

[0063] The attack and defense game model is a model based on game theory. Game theory is based on the idea that when two or more absolutely rational decision makers make decisions, they optimize their own strategies by guessing the strategies of other decision makers to maximize their own interests.

[0064] The attack-defense game model includes multiple game participants, their strategy space, signal space, prior probability, posterior probability and the benefits of each game participant. The model can be expressed as a six-tuple ADGMA = {P, A, S, p, p post , Reward}, where P = {Actor, Defender} represents the two parties involved in the game, which refers to the subject that can make autonomous decisions and behaviors in the game. In network attacks, it usually refers to the attacker and the defender. In unknown threat defense, the defender cannot determine the identity of the user, and the identity information of the defender is shared information. In the model, the participants are Actor and Defender, where Actor has two types Type = {Attacker, User}. When there are multiple Actors among the game participants, the control device assigns a corresponding and unique identifier to each Actor. For example: Actor1, Actor2.

[0065] A={A actor =(m 1 ,m 2 ,……,m x ),A defender =(a 1 ,a 2 ,……,a y )} is the strategy space of each participant, representing the specific set of actions that each participant can take, which can be observed by all participants and implies its type information. When there are multiple actors in the participant, each actor corresponds to a strategy space.

[0066] S={s1,s2,……,s n} represents the signal space, and the information about the true type conveyed by the Actor's behavior is called a "signal", which can be used to guide the judgment and correction of its type.

[0067] p represents the prior probability, which refers to the initial judgment of the defender that the attacker's actor type is Attacker, that is, p(Type=Attacker)=θ, p(Type=User)=1-θ.

[0068] p post stands for posterior probability, which refers to the posterior probability obtained by the defender after correcting the prior probability of the actor type judgment based on the acquired signal using the Bayesian rule.

[0069] Reward = {R A ,R D}Represents the net benefits that can be brought by the actions taken by each participant, which is usually related to the value and cost of the behavior and is the fundamental basis for the participants to choose their behavior.

[0070] After establishing the attack and defense game model, the control terminal initializes the model, where the prior probability is initialized to 0.5, the posterior probability is 0, and the net income of each participant is also 0.

[0071] S202: The control device repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior probability of the identity type of the attack terminal.

[0072] The control device is the defender in the attack and defense game model established in step S201, and the attack terminal is the attacker in the attack and defense game model.

[0073] The prior probability of the identity type of the attacking terminal is the prior probability of the identity type of the attacking terminal determined by the control device before the target working terminal receives the attack operation of the attacking terminal. If the target working terminal is attacked for the first time, the prior probability is the initialization value, that is, 0.5; otherwise, the prior probability is the posterior probability of the identity type of the attacking terminal determined by the control device when the target working terminal was attacked last time.

[0074] When the target working terminal determines that the access request it receives contains abnormal operations, it will feed back the abnormal status information of the access request to the control device. For network attacks, the attacking terminal divides the access request with attack behavior into multiple steps. There is also a certain causal relationship between each step. Each step of behavior can be an atomic behavior e. The atomic behavior forms an effective behavior m in time sequence and causal sequence. Then m=(e1,e2,……,e k ), where k is the number of atomic behaviors contained in the valid behavior m. When at least one atomic behavior in the valid behaviors received by the target working terminal is an abnormal behavior in the atomic abnormal behavior table, the valid behavior is determined to be abnormal. In one embodiment, the atomic abnormal behavior table is shown in Table 1.

[0075] Table 1

[0076]

[0077]

[0078] After determining that the target working terminal has received the attack operation, the control device updates the signal space of the attack and defense game model according to the attack strategy information it observes.

[0079] S203. The control device determines the optimal defense strategy and the optimal attack strategy for the target working terminal according to the attack strategy information and the prior probability, and executes the optimal defense strategy.

[0080] More specifically, the control device calculates the posterior probability of the identity type of the attack terminal according to the attack strategy information and the prior probability, and then determines the identity type of the attack terminal according to the posterior probability. The control device also calculates the attacker's benefit of the attack terminal and the defender's benefit of the target system according to the attack strategy information and the identity type, and determines the optimal defense strategy and the optimal attack strategy for the target working terminal according to the posterior probability and the defender's benefit.

[0081] More specifically, the control device calculates the posterior probability that the attacker's type is Attacker or User according to the Bayesian rule, and compares the two posterior probabilities, determines the type with the larger posterior probability value as the attacker's identity type, and determines the posterior probability corresponding to the type as the attacker's posterior probability. At the same time, the posterior probability is used to update the prior probability in the attack and defense game model, so that it serves as the prior probability when the target working terminal receives an attack again.

[0082] The control device first determines the system cost and type cost according to the identity type and attack strategy information of the attacker. The control device obtains the attack strategy information from the target working terminal and queries the system cost mapping table to obtain the system cost corresponding to the attack strategy information. The system cost indicates the impact on the benefits of other participants caused by changes in the system state after the attacker or defender implements the behavior.

[0083] The control device also queries the attack strategy information and the identity type determined according to the posterior probability in the type cost mapping table to obtain the type cost corresponding to the attack strategy information, which represents the cost required for different types of participants to choose behaviors. Attacker ∈[100,300],TC user ∈[500,2000].

[0084] More specifically, the system cost mapping table and the type cost mapping table can be found in the attack and defense behavior classification database of MIT's Lincoln Attack and Defense Laboratory, and will not be described in detail here.

[0085] The control device then calculates the attacker's benefit to the attack terminal and the defender's benefit to the target system based on the system cost, type cost, attack strategy information and posterior probability.

[0086] More specifically, the attacker's strategy information includes multiple atomic behaviors. The control device searches the atomic abnormal behavior table involved in step S202 for each atomic behavior, determines the self-value and potential value corresponding to each atomic behavior, and calculates the total value corresponding to each atomic behavior using the total value calculation formula, self-value, potential value and weight. The total value calculation formula is specifically expressed as:

[0087] r0(e)=SV(e)+PV(e)×weight,

[0088] Among them, r0(e) represents the total value of atomic behavior e, SV(e) represents the intrinsic value of atomic behavior e, PV(e) represents the potential value of atomic behavior e, and weight represents the weight of the potential value. Among them, SV(e) is related to the execution difficulty, resource capacity, and environmental requirements of atomic behavior e. The lower the execution difficulty, the higher the resource capacity, and the lower the environmental requirements, the higher the intrinsic value; PV(e) is related to factors such as the degree of publicity, defense maturity, and universality of e. The lower the degree of publicity, the lower the defense maturity, and the higher the universality, the higher the potential value. For example, the atomic behavior implemented by using the zero-day vulnerability in common office software has a very high potential attack value; weight∈[0,1] represents the degree to which the potential value of atomic behavior e plays a role, which is usually related to the level of atomic behavior e. The level of the atomic behavior represents the execution difficulty and effectiveness level of the atomic attack (defense) behavior, which is determined based on the attack and defense classification data of Lincoln Laboratory.

[0089] After obtaining the total value of each atomic behavior, the control device adds up the total values ​​corresponding to all atomic behaviors contained in the attack strategy information to obtain the effective behavior value corresponding to the attack strategy information. The control device subtracts the system cost and type cost from the effective behavior value to calculate the attacker's benefit to the attack terminal.

[0090] The control device obtains all the defense atomic behaviors that it can perform from the defense behavior table, and constructs a defense strategy information table for the attacker strategy information based on all the defense atomic behaviors, wherein there are more than one defense strategy information table. The control device calculates the effective behavior value corresponding to each defense strategy information table, and determines the difference between the effective behavior value and the system cost as the defender benefit corresponding to the defense strategy information table. In one embodiment, the defense behavior table is shown in Table 2:

[0091] Table 2

[0092]

[0093]

[0094] The control device calculates the optimal defense strategy according to the defender value and posterior probability corresponding to each defense strategy information table, and estimates the optimal attack strategy for the attack terminal according to the optimal defense strategy. The optimal defense strategy and the optimal attack strategy are obtained by solving game theory. The solution process is an existing technology and will not be repeated here.

[0095] S204. The control device calculates the Markov learning function value according to the optimal defense strategy and attack strategy information.

[0096] The Markov learning function value is the cumulative learning function value calculated using the Markov decision process from the initial state when the target working terminal is first attacked to the current state.

[0097] S205. The control device determines whether the target working terminal is in a safe state according to the Markov learning function value.

[0098] When the value of the Markov learning function is less than or equal to the safety threshold, it is determined that the target work terminal is in a safe state, and when the value of the Markov learning function is greater than the safety threshold, it is determined that the target work terminal is not in a safe state. In one embodiment, the safety threshold is 1000.

[0099] The security threshold represents a threshold at which no matter what attack strategy information is received, the target work terminal will not be seriously threatened when it is attacked next time.

[0100] When the target work terminal is in a safe state, enter step S202 to carry out the next round of attack and defense game, otherwise, enter step S206.

[0101] S206: Control the device to perform corresponding security operations.

[0102] In one embodiment, when performing security operations, the control device transfers the access request obtained by the target working terminal to the honeynet cluster, interrupting the operation of the attacking terminal on the target working device.

[0103] In the above technical solution, the control device establishes an attack and defense game model, and repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior type of the identity type of the attack terminal. After the operation, the optimal defense strategy and the optimal attack strategy for the target working terminal are determined according to the attack strategy information and the prior probability, and the optimal defense strategy is executed. The control device calculates the Markov learning function value according to the optimal defense strategy and the attack strategy information until it stops the loop when it is determined that the target working terminal is not in a safe state according to the Markov learning function value, so that the control device determines the attack state of the target working terminal according to the continuous attack strategy information obtained by the target working terminal, which helps it to determine the covert attack according to the continuously obtained information, so that it can better protect the target working terminal.

[0104] Figure 3 This is a schematic diagram of the calculation process of the Markov learning function value provided by this application according to an exemplary embodiment. Figure 3 As shown, the method includes:

[0105] S301. The control device calculates the attack and defense reward value of the target working terminal in the current state according to the optimal defense strategy and attack strategy information.

[0106] The strategy space of the game participants includes a defense strategy space, which includes at least one defense strategy; the strategy space of the game participants also includes an attack strategy space, which includes at least one attack strategy.

[0107] The control device calculates the attack and defense reward value of the target working terminal in the current state according to the optimal defense strategy, attack strategy information and the attack and defense reward value calculation formula, wherein the attack and defense reward value calculation formula specifically includes:

[0108]

[0109] Among them, s p Indicates the current state of the target work terminal, Re(f x (s p )) represents the attack and defense reward value of the target work terminal x in the current state, i represents the defense identifier in the defense strategy space, j represents the attack identifier in the attack strategy space, i=0 represents that the target work terminal x is not defended, i≠0 represents that the target work terminal x is defended by the i-th defense strategy, j=0 represents that the target work terminal x is not attacked, j≠0 represents that the target work terminal x is attacked by the j-th attack strategy, P represents the protection value, in one embodiment, P is a constant, D i represents the reward value when the target working terminal x is protected by the i-th defense strategy, δ irepresents the reward coefficient, D represents the attack reward value when the target working terminal is attacked without being protected by the defense strategy, and Pk ij It represents the reward value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

[0110] S302: The control device obtains the cumulative reward value of the target working terminal from the initial state to the current state and the Markov learning function value of the target working terminal in the current state.

[0111] The cumulative return value of the target working terminal from the initial state to the current state can be obtained by the return value accumulation formula, wherein the return value accumulation formula is specifically expressed as:

[0112]

[0113] in, Indicates that the target work terminal x changes from the current state s0 to state s p The cumulative return value at the time, Re(f x (s0)) represents the attack and defense reward value corresponding to the current state; s1 represents the first state of the target working terminal. The first state is the next state of the target terminal in the current state after it changes under the influence of the attack strategy and the protection strategy. Re(f x (s1)) represents the attack and defense reward value corresponding to the first state; s p represents the future p-th state of the current state of the terminal, Re(f x (s p )) represents the attack and defense reward value corresponding to state p; γ represents the discount rate parameter, γ∈[0,1]. When the discount rate is 0, the target system only considers the current reward; when the discount rate is 1, the target system will get long-term high rewards.

[0114] The learning rate parameter indicates the learning state of the control device during the state change process when the target working terminal is attacked. The learning rate parameter ∈ [0, 1], a learning rate parameter of 0 means that the subsequent behavior has no effect on the current; a learning parameter of 1 means that the impact of the subsequent behavior on the current should be fully considered.

[0115] The Markov learning function value of the target working terminal in the current state is the cumulative reward value obtained by the control device from the initial state when the target working device was attacked to the current state, wherein if the current state is the initial state when it was attacked for the first time, the Markov learning function in the current state is a preset initial value, and in one embodiment, the preset initial value is 0. More specifically, when the state of the target working terminal is p, the state of the target working device is a process of transforming from p-1 to p when it was attacked last time.

[0116] S303. The control device calculates the maximum cumulative reward value of the target working terminal in the next state according to the optimal attack strategy of the attack terminal for the optimal defense strategy, all defense strategies, cumulative reward values ​​and attack and defense reward values.

[0117] The control device calculates the attack and defense reward value of the target working terminal in the next state adjusted based on each defense strategy according to the optimal attack strategy of the attack terminal for the optimal defense strategy, each defense strategy and the attack and defense reward value formula involved in step S301, that is, based on the attack strategy to be adopted by the attack terminal predicted by the control device and all defense strategies that the control device can perform for the attack strategy, the attack and defense reward value corresponding to each defense strategy is calculated respectively.

[0118] After the control device obtains the attack and defense reward values, it adds each attack and defense reward value to the cumulative reward value obtained in step S302 to calculate the cumulative reward value of the target working terminal in the next state adjusted based on each defense strategy. The control device determines the maximum value of the cumulative reward values ​​in the next state corresponding to each defense strategy as the maximum cumulative reward value.

[0119] S304, the control device calculates the Markov learning function value of the target working terminal in the next state of the current state according to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state and the Markov learning function value in the current state.

[0120] According to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state, the Markov learning function value in the current state and the learning function update formula, the Markov learning function value of the target working terminal in the next state of the current state is calculated; wherein the learning function update formula specifically includes:

[0121]

[0122]

[0123] in, represents the Markov learning function value of the target working terminal in the next state of the current state, represents the Markov learning function value of the target working terminal in the current state, α represents the learning rate parameter, γ represents the discount rate parameter, represents the maximum cumulative reward value in the next state; s0 represents the initial state of the target work terminal, Re(f x (s0)) represents the attack and defense reward value corresponding to the initial state; s1 represents the first state of the target working terminal, which is the state of the target terminal in the initial state after it changes under the influence of the attack strategy and the protection strategy. x (s1)) represents the attack and defense reward value corresponding to the first state; sp Represents the current state of the terminal, Re(f x (s p )) represents the attack and defense reward value corresponding to the current state; s p+1 represents the next state of the target work terminal, Re(f x (s p+1 )) represents the attack and defense reward value corresponding to the next state. The initialization value of the Markov learning function value is 0.

[0124] In the above technical scheme, after the control device establishes the attack and defense game model, it obtains the prior probability and the attack strategy information observed by the control device in at least one cycle, and determines the optimal defense strategy of the control device against the attack terminal and the optimal attack strategy of the attack terminal for the optimal defense strategy according to the prior probability and the attack strategy information in each cycle, and uses the Markov learning function to determine whether the target working terminal is dangerous under the influence of this attack strategy, so that the control device terminates the cycle when it determines that the target working terminal is not in a safe state, so that the control device monitors multiple attacks performed by the attack terminal, and counts the damage caused by multiple attacks to the target system, and perceives the hidden and continuous attack operations, so that the control device can timely protect the target working terminal when it determines that the target working terminal is not in a safe state, and effectively ensure the safety of the target working terminal.

[0125] Figure 4 This is a schematic diagram of the structure of a network defense device provided according to an embodiment of the present application. The network defense device 400 includes an acquisition module 401 and a processing module 402, wherein:

[0126] The processing module 402 is used to establish an attack and defense game model; wherein the attack and defense game model includes multiple game participants, the strategy space, signal space, prior probability, posterior probability and the benefits of each game participant.

[0127] The acquisition module 401 is used to repeatedly execute to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior probability of the identity type of the attack terminal.

[0128] The processing module 402 is further used to determine the optimal defense strategy and the optimal attack strategy of the attack terminal for the optimal defense strategy according to the attack strategy information and the prior probability, and execute the optimal defense strategy;

[0129] The processing module 402 is also used to calculate the Markov learning function value according to the optimal defense strategy and attack strategy information;

[0130] The processing module 402 is further used to determine whether the target working terminal is in a safe state according to the Markov learning function value and terminate the loop when the target working terminal is not in a safe state.

[0131] In one embodiment, the processing module 402 is specifically configured to:

[0132] Calculate the posterior probability of the identity type of the attack terminal based on the attack strategy information and the prior probability;

[0133] According to the posterior probability, the identity type of the attacking terminal is determined;

[0134] Determine system cost and type cost based on attack strategy information and identity type;

[0135] Calculate the attacker's profit of attacking the terminal and the defender's profit of the target system based on the system cost, type cost, attack strategy information and posterior probability;

[0136] According to the posterior probability and the defender's benefit, the optimal defense strategy and the optimal attack strategy for the target working terminal are determined.

[0137] In one embodiment, the processing module 402 is specifically configured to:

[0138] Searching the attack strategy information in the system cost mapping table to obtain the system cost corresponding to the attack strategy information;

[0139] The attack strategy information and the identity type are queried in the type cost mapping table to obtain the type cost corresponding to the attack strategy information.

[0140] In one embodiment, the processing module 402 is specifically configured to:

[0141] According to the optimal defense strategy and attack strategy information, calculate the attack and defense reward value of the target working terminal in the current state;

[0142] Obtain the cumulative reward value of the target working terminal from the initial state to the current state and the Markov learning function value of the target working terminal in the current state;

[0143] Calculate the maximum cumulative reward value of the target working terminal in the next state according to the optimal attack strategy of the attack terminal for the optimal defense strategy, all defense strategies, cumulative reward values ​​and attack and defense reward values;

[0144] According to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state, and the Markov learning function value in the current state, the Markov learning function value of the target working terminal in the next state of the current state is calculated.

[0145] In one embodiment, the processing module 402 is specifically configured to:

[0146] According to the optimal defense strategy, attack strategy information and attack and defense reward value calculation formula, calculate the attack and defense reward value of the target working terminal in the current state:

[0147] The attack and defense reward value calculation formula specifically includes:

[0148]

[0149] Among them, s p Indicates the current state of the target work terminal, Re(f x (s p )) represents the attack and defense reward value of the target working terminal x in the current state, i represents the defense identifier in the defense strategy space, j represents the attack identifier in the attack strategy space, i=0 represents that the target working terminal x is not defended, i≠0 represents that the target working terminal x is defended by the i-th defense strategy, j=0 represents that the target working terminal x is not attacked, j≠0 represents that the target working terminal x is attacked by the j-th attack strategy, P represents the protection value, D i represents the reward value when the target working terminal x is protected by the i-th defense strategy, δ i represents the reward coefficient, D represents the attack reward value when the target working terminal is attacked without being protected by the defense strategy, and Pk ij It represents the reward value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

[0150] In one embodiment, the processing module 402 is specifically configured to:

[0151] According to the optimal attack strategy of the attack terminal for the optimal defense strategy, each defense strategy and the attack and defense reward value formula, the attack and defense reward value of the target working terminal in the next state adjusted based on each defense strategy is calculated;

[0152] Add each attack and defense reward value to the cumulative reward value, and calculate the cumulative reward value of the target working terminal in the next state adjusted based on each defense strategy;

[0153] The maximum value of the accumulated reward values ​​in the next state corresponding to each defense strategy is determined as the maximum accumulated reward value.

[0154] In one embodiment, the processing module 402 is specifically configured to:

[0155] According to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state, the Markov learning function value in the current state and the learning function update formula, the Markov learning function value of the target working terminal in the next state of the current state is calculated; wherein the learning function update formula specifically includes:

[0156]

[0157]

[0158] in, represents the Markov learning function value of the target working terminal in the next state of the current state, represents the Markov learning function value of the target working terminal in the current state, α represents the learning rate parameter, γ represents the discount rate parameter, represents the maximum cumulative reward value in the next state; s0 represents the initial state of the target work terminal, Re(f x (s0)) represents the attack and defense reward value corresponding to the initial state; s1 represents the first state of the target working terminal, which is the state of the target terminal in the initial state after it changes under the influence of the attack strategy and the protection strategy. x (s1)) represents the attack and defense reward value corresponding to the first state; s p Represents the current state of the terminal, Re(f x (s p )) represents the attack and defense reward value corresponding to the current state; s p+1 represents the next state of the target work terminal, Re(f x (s p+1 )) represents the attack and defense reward value corresponding to the next state.

[0159] In one embodiment, the processing module 402 is specifically configured to:

[0160] When the value of the Markov learning function is less than or equal to the safety threshold, it is determined that the target working terminal is in a safe state;

[0161] When the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in a safe state.

[0162] Figure 5 This is a schematic diagram of the structure of a control device provided according to an embodiment of the present application. The control device 500 includes a memory 501 and a processor 502, and the memory 501 is used to store computer instructions executable by the processor. The memory 501 may include a high-speed random access memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0163] When executing computer instructions, the processor 502 implements the various steps of the network defense method with the control device as the execution subject in the above embodiment. For details, please refer to the relevant description in the above method embodiment. The processor 502 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0164] Optionally, the memory 501 can be independent or integrated with the processor 502. When the memory 501 is independently provided, the control device 500 further includes a bus for connecting the memory 501 and the processor 502. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0165] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When a processor executes the computer instructions, each step of the network defense method in the above embodiment is implemented.

[0166] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the various steps in the network defense method in the above embodiment.

[0167] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0168] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A network defense method, characterized in that: The method is applied to a control device, the control device is located in a target system, the target system includes at least one working terminal and a honeynet cluster, and the method includes: Establishing an attack-defense game model; wherein the attack-defense game model includes a plurality of game participants, a strategy space of each of the game participants, a signal space, a priori probability, a posterior probability and the benefits of each of the game participants; the strategy space of the game participants includes a defense strategy space, and the defense strategy space includes at least one defense strategy; Repeatedly obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the prior probability of the identity type of the attack terminal; Determine, according to the attack strategy information and the prior probability, an optimal defense strategy and an optimal attack strategy of the attack terminal for the optimal defense strategy, and execute the optimal defense strategy; Calculate the attack and defense reward value of the target working terminal in the current state according to the optimal defense strategy and the attack strategy information; Obtaining a cumulative reward value of the target work terminal from an initial state to a current state and a Markov learning function value of the target work terminal in the current state; Calculate the maximum cumulative reward value of the target working terminal in the next state according to the optimal attack strategy of the attack terminal for the optimal defense strategy, all the defense strategies, the cumulative reward value and the attack and defense reward value; According to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state and the Markov learning function value in the current state, the Markov learning function value of the target work terminal in the next state of the current state is calculated; according to the Markov learning function value, it is determined whether the target work terminal is in a safe state and the loop is terminated when the target work terminal is not in a safe state.

2. The method according to claim 1, characterized in that Determining an optimal defense strategy and an optimal attack strategy for the target working terminal according to the attack strategy information and the prior probability, specifically including: Calculating the posterior probability of the identity type of the attacking terminal according to the attack strategy information and the prior probability; Determining the identity type of the attacking terminal according to the posterior probability; Determining a system cost and a type cost according to the attack strategy information and the identity type; Calculating the attacker benefit of the attack terminal and the defender benefit of the target system according to the system cost, the type cost, the attack strategy information and the posterior probability; An optimal defense strategy and an optimal attack strategy for the target working terminal are determined according to the posterior probability and the defender benefit.

3. The method according to claim 2, characterized in that According to the attack strategy information and the identity type, a system cost and a type cost are determined, specifically including: Searching the attack strategy information in a system cost mapping table to obtain a system cost corresponding to the attack strategy information; The attack strategy information and the identity type are searched in a type cost mapping table to obtain a type cost corresponding to the attack strategy information.

4. The method according to claim 1, characterized in that: The strategy space of the game participants also includes an attack strategy space, and the attack strategy space includes at least one attack strategy; according to the optimal defense strategy and the attack strategy information, the attack and defense reward value of the target working terminal in the current state is calculated, specifically including: According to the optimal defense strategy, the attack strategy information and the attack and defense reward value calculation formula, the attack and defense reward value of the target working terminal in the current state is calculated: The attack and defense reward value calculation formula specifically includes: ， in, represents the pth state after the current state of the target working terminal, represents the attack and defense reward value of the target working terminal x in state p, i represents the defense identifier in the defense strategy space, j represents the attack identifier in the attack strategy space, Indicates that the target working terminal x is not protected in state p, indicates that the target working terminal x is protected by the i-th defense strategy when in state p, j=0 indicates that the target working terminal x is not attacked when in state p, It means that the target working terminal x is attacked by the jth attack strategy when in state p, P represents the protection value, represents the reward value when the target working terminal x is protected by the i-th defense strategy, represents the reward coefficient, D represents the attack reward value when the target working terminal is attacked without being protected by the defense strategy, It represents the reward value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

5. The method according to claim 4, characterized in that Calculating the maximum cumulative reward value of the target working terminal in the next state according to the optimal attack strategy of the attack terminal for the optimal defense strategy, all the defense strategies, the cumulative reward value and the attack and defense reward value, specifically including: According to the optimal attack strategy of the attack terminal for the optimal defense strategy, each of the defense strategies and the attack and defense reward value formula, calculate the attack and defense reward value of the target working terminal in the next state adjusted based on each of the defense strategies; Adding each of the attack and defense reward values ​​to the cumulative reward value, and calculating the cumulative reward value of the target working terminal in the next state adjusted based on each of the defense strategies; The maximum value of the accumulated reward values ​​in the next state corresponding to each of the defense strategies is determined as the maximum accumulated reward value.

6. The method according to claim 5, characterized in that Calculating the Markov learning function value of the target working terminal in the next state of the current state according to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state and the Markov learning function value in the current state, specifically includes: According to the attack and defense reward value in the current state, the maximum cumulative reward value in the next state, the Markov learning function value in the current state and the learning function update formula, the Markov learning function value of the target working terminal in the next state of the current state is calculated; wherein the learning function update formula specifically includes: ， , in, represents the Markov learning function value of the target working terminal in the next state of the current state, represents the Markov learning function value of the target working terminal in the current state, represents the learning rate parameter, represents the discount rate parameter, Indicates the maximum cumulative reward value in the next state; represents the initial state of the target working terminal, Indicates the attack and defense reward value corresponding to the initial state; Indicates the status of the target work terminal The next state of Indicates status The corresponding attack and defense reward value; Indicates the current state of the target work terminal, Indicates the attack and defense reward value corresponding to the current state; represents the next state of the current state of the target work terminal, Indicates the attack and defense reward value corresponding to the next state.

7. The method according to claim 1, characterized in that Determining whether the target working terminal is in a safe state according to the Markov learning function value specifically includes: When the Markov learning function value is less than or equal to the safety threshold, determining that the target working terminal is in the safety state; When the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in the safety state.

8. A control device, characterized in that: include: A processor and a memory communicatively connected to the processor; The memory stores computer instructions; When executing the computer instructions, the processor is used to implement the network defense method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed by a processor, are used to implement the network defense method according to any one of claims 1 to 7.

Images