An adaptive differential evolution dynamic honeypot strategy optimization method

Abstract

The present invention belongs to the field of network security, and relates to an adaptive differential evolution dynamic honeypot strategy optimization method, including: constructing a dynamic honeypot service population according to the service characteristics of a dynamic honeypot system, and randomly initializing the population; calculating the total value of honeypot services and real services as the fitness of the population according to the Cobb-Douglas utility function; using the Q-learning algorithm to adaptively adjust the mutation probability factor F and the crossover probability factor CR; using the differential evolution algorithm to perform mutation and crossover operations on individuals in the population; comparing the fitness values ​​of individuals in the population, and selecting better individuals to enter the next generation; and selecting the optimal service switching strategy for the dynamic honeypot through repeated iterations until the end. The present invention optimizes the service switching strategy of the dynamic honeypot system, takes the defense success rate of the honeypot service and the success rate of attacking the real service as indicators for evaluating the quality of the service, and uses the adaptive differential evolution algorithm to evolve the service switching strategy of the dynamic honeypot system, optimizes the traditional random service switching strategy, and makes the service switching of the dynamic honeypot system more in line with the actual defense needs of the network system, and significantly improves the security defense capability of the system.

Description

Technical Field

[0001] The invention belongs to the field of network security, and in particular relates to an adaptive differential evolution dynamic honeypot strategy optimization method. Background Art

[0002] With the rapid development of information technology, the security threats faced by network systems are increasing day by day. The inherent resource and information asymmetry in network attack and defense always puts the defender in a passive position. Since traditional passive defense technology cannot resist diverse and hidden network attack methods, active defense technology has gradually been applied to solve network attack and defense problems in recent years.

[0003] Dynamic honeypot is a typical active defense technology that resists malicious attacks through collaborative deception between host clusters. In a dynamic honeypot, each host deploys multiple services, and each service contains two attributes, real and fake. The dynamic honeypot system increases the uncertainty of attackers by constantly changing the true and false attributes of services between collaborative hosts, effectively protecting the hidden real services and resisting malicious attacks on the network system. However, the host cluster of the dynamic honeypot system obtains the specific information of service attribute switching by executing a random role allocation algorithm, and then the host cluster performs the service opening or closing operation according to the service switching information. The random service role allocation algorithm can allocate resources efficiently and fairly, avoiding the greedy problem of policy selection. However, the randomly assigned service attributes cannot effectively target malicious access in the system. When facing persistent threats, the security and stability of the dynamic honeypot system cannot be guaranteed.

[0004] In order to improve the security defense capability of the dynamic honeypot system, the defense strategy decision-making ability of the system should be strengthened so that the defense strategy executed by the system can effectively resist malicious attacks. At present, the defense decision research on dynamic honeypot systems mainly focuses on the defense decision research of dynamic honeypots based on game theory and the defense decision research of dynamic honeypots based on attack graphs. These studies all give defense strategies from a macro perspective through attack and defense information, and do not consider the optimization of the underlying algorithm.

[0005] Differential evolution algorithm is an intelligent evolutionary search algorithm generated by cooperation and competition among individuals in a group. The algorithm uses the individual differences in the current group to construct the differential mutation pattern of the mutant individuals. Due to the unknown and changeable nature of the network attack and defense environment, the search for the defense strategy space using the differential evolution method is dynamic. The use of fixed control parameters cannot meet the computational requirements of the dynamic optimization process of the strategy. Q-learning is a typical reinforcement learning algorithm that regards learning as a "try-evaluation" process. The intelligent agent continuously explores and attempts to establish a reward and punishment mechanism for the unknown environment to reward or punish each action of the intelligent agent, and then gradually forms action expectations, and realizes the optimal strategy selection under the Markov decision process through an iterative mechanism. If the reward and punishment mechanism of the Q-learning algorithm is used to adaptively adjust the control parameter set, that is, the mutation probability factor F and the crossover probability factor CR, the global search ability and convergence speed of the algorithm can be balanced. Therefore, the adaptive differential evolution algorithm is introduced into the dynamic honeypot system, and the service switching strategy of the host cluster is adaptively differentially evolved to obtain the optimal service switching strategy, so that the dynamic honeypot system can resist malicious attacks in complex network environments. Summary of the invention

[0006] Aiming at the problem that the random service switching strategy of the dynamic honeypot system cannot resist sustainable attacks, the present invention provides an adaptive differential evolution dynamic honeypot strategy optimization method, including: constructing a dynamic honeypot service population according to the service characteristics of the dynamic honeypot system, and randomly initializing the population; calculating the total value of the honeypot service and the real service as the population fitness according to the Cobb-Douglas utility function; using the Q-learning algorithm to adaptively adjust the mutation probability factor F and the crossover probability factor CR; using the differential evolution algorithm to perform mutation and crossover operations on individuals in the population; comparing the fitness values ​​of the individuals in the population, and selecting better individuals to enter the next generation; and selecting the optimal service switching strategy of the dynamic honeypot through repeated iterations until the end. The present invention uses an adaptive differential evolution algorithm to evolve the service switching strategy of the dynamic honeypot system, optimizes the traditional random service switching strategy, makes the service switching of the dynamic honeypot system more in line with the actual network defense needs, and significantly improves the security defense capability of the system.

[0007] The technical solution adopted by the present invention to solve its technical problem is:

[0008] An adaptive differential evolution dynamic honeypot strategy optimization method, the method comprising the following steps:

[0009] a. Construct dynamic honeypot service population X according to the service characteristics of dynamic honeypot system NP , the different individuals in the population are represented by X i =(x i1 ,x i2,...,x iD ),i=1,2,...,NP, and randomly initialize the individuals in the population as:

[0010] x i,j =x min,j +rand(0,1)·[x max,j -x min,j ],j=1,2,...,D

[0011] Among them, NP is the population size, D is the parameter space dimension of the service, and x min,j is the lower bound of the j-th dimension parameter, x max,j is the upper bound of the j-th dimension parameter, and rand(0,1) is a random number;

[0012] b. Calculate the total value of the honeypot service and the real service as the population fitness based on the Cobb-Douglas utility function

[0013] The Cobb-Douglas utility function can evaluate the relative importance of the honeypot service and the real service to the security of the host of the dynamic honeypot system, and the fitness evaluation function is:

[0014] U(x,y)=x α ·y β ,α∈[0,1],β∈[0,1],α+β=1

[0015] Among them, x is the honeypot service, y is the real service, α is the defense success rate of the honeypot service, that is, the probability that the honeypot service is accessed by an attacker; β is the attack success rate of the real service, that is, the probability that the real service is accessed by an attacker;

[0016] c. Adopt Q-learning algorithm to adaptively adjust the mutation probability factor F and crossover probability factor CR. Specifically, in each iteration, the optimal performance of the mutation probability factor F and crossover probability factor CR parameter set is selected according to the accumulated Q value table information.

[0017] In each iteration, the agent is given a state. After executing the current action, the agent obtains an instantaneous benefit and transfers to the next state. The Q value can be obtained by calculating the overall expected benefit of the agent in each iteration. Then, after each iteration, the Q value table information is updated and a reward is given to indicate the performance of the most recent iteration. The agent executes the optimal F and CR parameter set based on the accumulated Q value table information. The benefit function Q(·), the update formula of the Q value table, and the reward mechanism are respectively as follows:

[0018]

[0019] Q t+1 =(1+α)Q(s t ,at )+α[r t +γmax a Q(s t+1 ,a t )]

[0020]

[0021] Among them, s t is the state at time t, s t+1 is the state at time t+1, a t is the action executed at time t, p∈(0,1) is the state transition probability, γ is the discount factor, r(·) is the instantaneous benefit of the agent, α is the learning rate, r t is the reward and punishment value at time t, Q t+1 is the overall expected benefit of the agent at time t+1, that is, the Q value at time t+1, best t is the optimal Q value at time t, best t+1 is the optimal Q value at time t+1;

[0022] d. Use the differential evolution algorithm to perform mutation and crossover operations on individuals in the population. Specifically, the differential evolution mutation scheme selects vectors by combining random selection rand and best selection best, and the mutation scheme contains 1 differential vector. This scheme converts the population individual x i,j Mutation to v i,j , and perform crossover operation on the mutated individuals to obtain u i,j

[0023] The mutation scheme uses three randomly selected different individuals and one best individual to perform differential operations to interfere with an existing individual to achieve mutation. The formula is as follows:

[0024]

[0025] in, and are three different individuals randomly selected from the current population, x best is the best individual based on fitness value, v i,j is the mutant individual, F is the mutation probability factor;

[0026] The crossover operation transforms the population individuals x i,j and mutant individual v i,j Perform crossover operation to generate test individual u i,j To obtain greater population diversity, the crossover formula is:

[0027]

[0028] Among them, j rand represents a random integer to ensure that at least one dimension of the individual after crossover is provided by the mutant individual, rand(0,1) is a random number, and CR is the crossover probability factor;

[0029] e. Compare population individuals x i,j and the experimental individual u i,j The fitness value of , selects better individuals to enter the next generation, and the selection formula is as follows:

[0030]

[0031] Among them, g is the current evolutionary generation, g+1 is the next generation, and U(·) is the fitness value;

[0032] f. Iterate step c to step e. When i ≥ NP, the iteration ends and the optimal solution of the population is the optimal service switching strategy of the dynamic honeypot.

[0033] The present invention is an adaptive differential evolution dynamic honeypot strategy optimization method, which has the following advantages compared with the prior art:

[0034] An adaptive differential evolution algorithm is used to evolve the service switching strategy of the dynamic honeypot system. The Cobb-Douglas utility function is used as the fitness evaluation function to evaluate the relative importance of real services and honeypot services to the security of the system host, so that the system can obtain a defense strategy that is more in line with actual security needs. In each iteration, the Q-learning algorithm is used to adaptively adjust the mutation probability factor F and the crossover probability factor CR to meet the computing requirements of the dynamic optimization process, improve the overall performance of the algorithm, enable the system to obtain a more optimized service switching strategy, and improve the security defense capability of the dynamic honeypot system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution in the embodiment of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation plans:

[0036] Figure 1 It is a flow chart of an adaptive differential evolution dynamic honeypot strategy optimization method. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with embodiments and accompanying drawings, but the present application is not limited to these embodiments.

[0038] This embodiment provides an adaptive differential evolution dynamic honeypot strategy optimization method, the method comprising the following steps:

[0039] a. Construct a dynamic honeypot service population based on the service characteristics of the dynamic honeypot system and randomly initialize the population

[0040] In step a, each host of the dynamic honeypot system deploys multiple services, each service contains two attributes: true and false. By constantly changing the true and false attributes of the services between the host clusters working together, the uncertainty of the attacker can be increased. If the dynamic honeypot system is regarded as an evolving ecological environment, the different services deployed by the host are different populations. Assume that the service population size of the dynamic honeypot system is NP, and the individual parameter space in the population, i.e., the service attribute space, is D-dimensional, then the population X NP There are NP different individuals X i Represented as X i ={x i,1 ,x i,2 ,...,x i,D |i=1,2,...,NP}, each individual in the population is randomly initialized as:

[0041] x i,j =x min,j +rand(0,1)·[x max,j -x min,j ],j=1,2,...,D

[0042] Among them, x min,j is the lower bound of the j-th dimension parameter, x max,j is the upper bound of the j-th dimension parameter, and rand(0,1) is a random number.

[0043] b. Calculate the total value of the honeypot service and the real service as the population fitness based on the Cobb-Douglas utility function

[0044] In step b, the dynamic honeypot system continuously changes the service attributes provided by the host cluster to confuse attackers by evaluating network attack and defense information. The Cobb-Douglas utility function can evaluate the relative importance of real services and honeypot services to the security of the dynamic honeypot system host. The fitness evaluation function is:

[0045] U(x,y)=x α ·y β ,α+β=1

[0046] Among them, x is the honeypot service, y is the real service, α∈[0,1] is the defense success rate of the honeypot service, that is, the probability that the honeypot service is accessed by an attacker, and β∈[0,1] is the attack success rate of the real service, that is, the probability that the real service is accessed by an attacker.

[0047] c. Adaptively adjust the mutation probability factor F and the crossover probability factor CR according to the Q-learning algorithm

[0048] In step c, the real network attack and defense environment is complex and changeable. The defense strategy space of the dynamic honeypot system searched by the differential evolution method is dynamic. At each iteration, the control parameters in the differential evolution process need to be adaptively adjusted to meet the computing requirements of the dynamic optimization process of the strategy. The Q-learning algorithm is used to adaptively adjust the mutation probability factor F and the crossover probability factor CR, and various parameter sets of the agent are given. The agent is given a state at each iteration of the individual. After the agent executes the current action, it obtains an instantaneous benefit and transfers to the next state. The Q value can be obtained by calculating the overall expected benefit of the intelligent agent at each iteration. Then, after each iteration, the Q value table information is updated and a reward is given to indicate the performance of the most recent iteration. The agent executes the optimal F and CR parameter sets based on the accumulated Q value table information. The benefit function Q(·), the update formula of the Q value table, and the reward mechanism are shown as follows:

[0049]

[0050] Q t+1 =(1+α)Q(s t ,a t )+α[r t +γmax a Q(s t+1 ,a t )]

[0051]

[0052] Among them, s t is the state at time t, s t+1 is the state at time t+1, a t is the action executed at time t, p∈(0,1) is the state transition probability, γ is the discount factor, r(·) is the instantaneous benefit of the agent, α is the learning rate, r t is the reward and punishment value at time t, Q t+1 is the overall expected benefit of the agent at time t+1, that is, the Q value at time t+1, best t is the optimal Q value at time t, best t+1 is the optimal Q value at time t+1.

[0053] d. Use differential evolution algorithm to perform mutation and crossover operations on individuals in the population

[0054] In step d, the differential evolution mutation scheme selects vectors by combining random selection rand and best selection best, and the mutation scheme includes 1 differential vector. Specifically, the scheme uses three different individuals and one best individual to perform differential operation to interfere with an existing individual so that it can achieve population individual xi,j Mutation to v i,j , the mutation formula is:

[0055]

[0056] in, and are three different individuals randomly selected from the current population, x best is the best individual based on fitness value, v i,j is the mutant individual, F is the mutation probability factor;

[0057] In step d, the population individual x i,j and mutant individual v i,j Perform crossover operation to generate test individual u i,j To obtain greater population diversity, the crossover formula is:

[0058]

[0059] Among them, j rand Represents a random integer, ensuring that at least one dimension of the individual after crossover is provided by the mutant individual, rand(0,1) is a random number, and CR is the crossover probability factor.

[0060] e. Compare population individuals x i,j and the experimental individual u i,j The fitness value of , selects better individuals to enter the next generation, and the selection formula is as follows:

[0061]

[0062] Among them, g is the current evolutionary generation, g+1 is the next generation, and U(·) is the fitness value.

[0063] f. Iterate step c to step e. When i ≥ NP, the iteration ends and the optimal solution of the population is the optimal service switching strategy of the dynamic honeypot.

Claims

1. An adaptive differential evolution dynamic honeypot strategy optimization method, characterized in that: The following steps are involved: a. Construct dynamic honeypot service population X according to the service characteristics of dynamic honeypot system NP , the different individuals in the population are represented by X i =(x i1 ,x i2 ,...,x iD ),i=1,2,...,NP, and randomly initialize the individuals in the population as: x i,j =x min,j +rand(0,1)·[x max,j -x min,j ],j=1,2,...,D Among them, NP is the population size, D is the dimension of the service parameter space, x min,j is the lower bound of the j-th dimension parameter, x max,j is the upper bound of the j-th dimension parameter, and rand(0,1) is a random number; b. Calculate the total value of the honeypot service and the real service as the population fitness according to the Cobb-Douglas utility function: U(x,y)=x α ·y β ,α∈[0,1],β∈[0,1],α+β=1 Among them, x is the honeypot service, y is the real service, α is the defense success rate of the honeypot service, that is, the probability that the honeypot service is accessed by an attacker, and β is the attack success rate of the real service, that is, the probability that the real service is accessed by an attacker; c. Adopt Q-learning algorithm to adaptively adjust the mutation probability factor F and the crossover probability factor CR. Specifically, in each iteration, the parameter set of the mutation probability factor F and the crossover probability factor CR with the best performance is selected according to the accumulated Q value table information. The update formula of the Q value table is: Q t+1 =(1+α)Q(s t ,a t )+α[r t +γmax a Q(s t+1 ,a t )] Among them, s t is the state at time t, s t+1 is the state at time t+1, a t is the action executed at time t, α is the learning rate, γ is the discount factor, r t is the reward and punishment value at time t, Q t+1 is the overall expected return of the agent at time t+1, that is, the Q value at time t+1, and Q(·) is the return function; d. Use the differential evolution algorithm to perform mutation and crossover operations on individuals in the population. Specifically, the differential evolution mutation scheme selects vectors by combining random selection rand and best selection best, and the mutation scheme contains 1 differential vector. This scheme converts the population individual x i,j Mutation to v i,j , and perform crossover operation on the mutated individuals to obtain the test individual u i,j , the mutation formula and crossover formula are as follows: in, and are three different individuals randomly selected from the current population, x best is the best individual based on fitness value, j rand represents a random integer to ensure that at least one dimension of the individual after crossover is provided by the mutant individual, v i,j is the mutant individual, rand(0,1) is a random number, F is the mutation probability factor, and CR is the crossover probability factor; e. Compare population individuals x i,j and the experimental individual u i,j The fitness value of , selects better individuals to enter the next generation, and the selection formula is as follows: Among them, g is the current evolutionary generation, g+1 is the next generation, and U(·) is the fitness value; f. Iterate step c to step e. When i ≥ NP, the iteration ends and the optimal solution of the population is the optimal service switching strategy of the dynamic honeypot.

Images