A network anomaly detection method based on swarm intelligence algorithm and isolated forest

Abstract

This invention discloses a swarm intelligence optimization algorithm based on a cultural framework and a network anomaly detection method using isolated forests. The method includes: randomly dividing a dataset into training and testing sets; constructing an isolated forest model for network anomaly detection based on the training set; fusing and optimizing an artificial fish swarm algorithm and a predator algorithm to generate a new swarm intelligence optimization algorithm based on a cultural framework; optimizing the isolated forest model using the swarm intelligence optimization algorithm, and selecting isolated trees with good detection capabilities to participate in the ensemble; and validating network anomaly detection on the testing set. To compensate for the shortcomings of the isolated forest algorithm, the optimal subset is found from the already generated isolated tree set in the training set using the artificial fish swarm algorithm and the predator algorithm within a cultural framework. This allows the isolated forest algorithm to improve network anomaly identification accuracy while reducing the number of isolated trees, thereby increasing its running speed.

Description

Technical Field

[0001] This invention relates to the field of network anomaly detection technology, specifically to a network anomaly detection method based on swarm intelligence algorithms and isolated forests. Background Technology

[0002] Existing network fault diagnosis techniques build models based on normal data and then classify data that does not conform to the model as anomalies, which suffers from drawbacks such as computational complexity and poor performance. Isolation forests, on the other hand, utilize the characteristic that anomalous data constitutes a small proportion and deviates from the overall population to quickly separate them from normal data. Therefore, the algorithm itself has advantages such as lower linear time complexity and lower memory requirements. Despite its advantages in anomaly detection algorithms, isolation forests still have some issues that need improvement: the high detection accuracy of isolation forests relies on the ensemble of a large number of isolation trees, and training a large number of isolation trees will reduce the algorithm's efficiency and increase memory consumption; in isolation forests, each isolation tree has different detection capabilities, but they all have the same voting weight, so some trees with insufficient detection capabilities may negatively impact the final detection results.

[0003] In view of the above-mentioned defects, the inventors of this invention have finally obtained this invention after a long period of research and practice. Summary of the Invention

[0004] The purpose of this invention is to improve the convergence speed of traditional swarm intelligence optimization and make up for its tendency to get trapped in local optima. It integrates artificial fish swarms with predator algorithms to improve the detection accuracy of traditional isolated forest algorithms, thereby improving the network anomaly detection rate. This invention provides a network anomaly detection method for swarm intelligence algorithms and isolated forests within a cultural framework.

[0005] To achieve the above objectives, this invention provides a network anomaly detection method based on swarm intelligence algorithms and isolated forests, comprising the following steps:

[0006] S1: Training the model

[0007] The dataset is randomly divided into training and testing sets, and an isolated forest model for network fault detection is constructed based on the training set.

[0008] S2: Swarm Intelligence Optimization Algorithm

[0009] A novel swarm intelligence optimization algorithm is obtained by fusing and optimizing the artificial fish swarm algorithm and the predator algorithm based on the culture algorithm:

[0010] The cultural algorithm includes a population space and a belief space. Individuals in the population space evolve according to the predator algorithm mechanism, while individuals in the belief space evolve according to the artificial fish swarm algorithm mechanism.

[0011] The population space and the belief space are linked together through a communication protocol composed of acceptance functions and influence functions;

[0012] The population space contributes elite individuals to the faith space, where the elite individuals refer to the optimal solutions generated during the evolution of the predator algorithm in the population space.

[0013] After evolution, the faith space sends superior individuals into the population space to replace the inferior individuals. The superior individuals are those with high recognition accuracy generated during the evolution of the artificial fish swarm algorithm in the faith space, while the inferior individuals are those with low recognition accuracy generated during the evolution of the predator algorithm in the population space.

[0014] S3: Model Optimization

[0015] The isolated forest model is optimized using a swarm intelligence optimization algorithm, with the anomaly detection rate used as the fitness function.

[0016] The swarm intelligence optimization algorithm selects high-accuracy isolated trees from the isolated forest model to participate in the ensemble, and removes isolated trees with low accuracy; finally, the algorithm is tested using a test set.

[0017] Furthermore, step S1 specifically includes the following steps:

[0018] S11: First, collect the dataset for network anomaly detection, and divide the dataset into training set and test set; randomly select n sample data points from the training set as a sub-sample set, construct an isolation tree, and put the sub-sample set into the root node of the isolation tree;

[0019] A feature w is randomly selected from the subsample set, and then a cutoff value p is randomly selected. The cutoff value p is between the upper and lower values ​​of the selected feature w.

[0020] The sample data is divided into two parts, left and right, by the splitting value p. If di(w) > p, the data di(w) is divided into the right subtree, and vice versa.

[0021] Recursively construct new child nodes until only one data item in a child node cannot be further cut, at which point the isolation tree has reached the initially defined limit height.

[0022] S12: Construct n isolated trees as described above to form an isolated forest model;

[0023] S13: Select test data x in the test set and substitute it into each isolated tree in the isolated forest model;

[0024] x represents the test data. Calculate the path length h(x) that falls on each isolated tree, and calculate the average value E(h(x)) of all h(x).

[0025] The path length c(α) of the test data x can be obtained using the following formula. Where H(i)=Ln(i)+γ is the harmonic number, γ is the Euler constant, α is the number of leaf nodes, and c(α) is the average value of h(x) when α is determined;

[0026] The anomaly score s(x,α) of the test data x is calculated using the following formula:

[0027] Based on the magnitude of the anomaly score s(x,α), outliers and normal points are distinguished, and the abnormal data and normal data in the test set are separated.

[0028] Furthermore, the cultural algorithm in S2 for the fusion and optimization of the predator algorithm and the artificial fish swarm algorithm specifically includes the following steps:

[0029] The belief space is updated by selecting the better individuals in the current population space through the accept operation. Each time AcceptStep is run, the worst individual in the belief space is replaced with the current global best value in the population space. The calculation formula of AcceptStep is as follows.

[0030]

[0031] Where: AcceptStep is the number of accept operations, fix is ​​the rounding function, MAXIt is the maximum number of generations of population evolution preset, It is the current generation of population evolution, BaseNum and DevNum are freely set constants;

[0032] When It%AcceptStep=0, the worst individual in the belief space is replaced with the current global best value in the population space;

[0033] By influencing the behavior, superior individuals from the evolved belief space are sent into the population space to replace inferior individuals.

[0034]

[0035] Where: AffecttStep is the number of influencing operations, fix is ​​the rounding function, MAXIt is the maximum number of generations of population evolution preset by the population, It is the current generation of population evolution, BaseNum and DevNum are freely set constants; when It%AffecttStep=0, the worst individual in the population space is replaced by the current global optimal value in the belief space.

[0036] Furthermore, the artificial fish swarm algorithm in S2 includes the following steps:

[0037] S201: Initialization settings, including population size, initial positions of each artificial fish, vision of the artificial fish, step size, crowding factor, and number of repetitions;

[0038] S202: Bulletin board assignment: Calculate the fitness values of each individual in the initial fish swarm, and assign the optimal artificial fish state and its fitness value to the bulletin board;

[0039] S203: Behavior selection: Evaluate each individual and select the behavior it will execute, including foraging, schooling, following, and random behaviors;

[0040] S204: Execute the behavior of the artificial fish and update to generate a new fish swarm;

[0041] S205: Bulletin board update: Evaluate all individuals. If the fitness value of an individual is better than that of the bulletin board, update the bulletin board to this fitness value;

[0042] S206: When the fitness value on the bulletin board reaches the global optimal solution of the algorithm or reaches the upper limit of the iteration number, the algorithm ends; otherwise, execute step S203.

[0043] Furthermore, the predator algorithm in S2 includes the following steps:

[0044] S211: Initialization stage, the predator randomly initializes its position x within the search range i ;

[0045] S212: Predator search stage, find the next iteration position x(t + 1) of the predator.

[0046] Furthermore, the initial position x in S211 i is calculated by the following formula: x i = rand(1, d).*(ub - lb)+lb, 0 < i ≤ N; where x i is the initial position of the predator, lb is the minimum value of the problem variable, ub is the maximum value of the problem variable, d is the number of problem variables, and the problem variable refers to the solution space vector;

[0047] To expand the search range of the solution, the idea of reverse competition is introduced: Let x ∈ [lb, ub] be a real number, then its corresponding global reverse number is defined as: 2N initial solutions are obtained through the above operations.

[0048] Furthermore, S212 includes the following steps:

[0049] S2121: Find the next iteration position x(t + 1) of the predator, x ij (t + 1)= x ij (t)+0.5[(2CZPpos(j) -x ij (t))+(2(1-C)Zμ(j)-x ij [(t))], where x(t) is the current predator position, P pos Z is the position of the prey, μ is the average of the predator's position and the prey's position, Z is the adaptive parameter, and C is the equilibrium parameter.

[0050] S2122: Locate the prey Where kbest = round(C × 2N), N is the number of search agents Z, and kbest is the position of the prey search agent;

[0051] S2123: Find the updated location of the prey x ij (t+1), x ij (t+1)=T pos(j) +R ij (t)(T pos(j) -x ij x(t)); where x(t) is the current prey position, x(t+1) is the prey's next iteration position, and T pos It is the globally optimal position; R ij It is a random number in the range [0,1] used to select predators and prey;

[0052] S2124: Combining S2121, S2122, and S2123, obtain the next iteration position x of the prey and the predator. i The formula for (t+1):

[0053]

[0054] Furthermore, the adaptive parameter Z in S2121 is calculated using the following formula:

[0055]

[0056] in and Let P be a random vector within the range [0,1]. The index value, R2 is a random number in [0,1], and IDX is a vector that satisfies the condition P==0. The index value.

[0057] Furthermore, the balance parameter C in S2121 is calculated using the following formula: Where it is the current iteration number, and MaxIt is the maximum iteration number;

[0058] The positional average value μ is calculated using the following formula:

[0059] Furthermore, in S2122, the distance D between each predator and the average position μ is... edu(i) Calculated using the following formula Where μ is the positional average.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0061] 1. We propose an artificial fish swarm and predator algorithm under a cultural framework, which improves the convergence speed of traditional swarm intelligence optimization algorithms, has parallel processing capabilities, faster optimization speed, global optimization capabilities, and can quickly escape local extreme points.

[0062] 2. The artificial fish swarm algorithm has a strong global search capability, while the predator algorithm has a strong local optimization capability. To overcome the shortcomings of a single algorithm, the culture algorithm combines them into a faith space and a population space, forming a new multi-population intelligent collaborative algorithm. Individuals in the population space evolve according to the predator algorithm mechanism, while individuals in the faith space evolve according to the artificial fish swarm algorithm mechanism. The two spaces are independent yet interconnected, linked by a communication protocol composed of acceptance and influence functions. The lower population space periodically contributes elite individuals to the upper faith space. After evolution, the upper faith space sends some of its better individuals into the population space to replace the worse individuals, thus forming a collaborative mechanism of "dual evolution and dual promotion," thereby improving population diversity and algorithm efficiency.

[0063] 3. Use artificial fish swarms and predator algorithms under a cultural framework to optimize isolated forests and improve the accuracy of network device fault diagnosis. Attached Figure Description

[0064] Figure 1 This is a schematic diagram illustrating the principle of the fusion of artificial fish swarms and predator algorithms based on a cultural framework, as described in this invention.

[0065] Figure 2 This is a schematic diagram illustrating the principle of optimizing isolated forests using a swarm intelligence algorithm based on a cultural framework, as described in this invention.

[0066] Figure 3 This is a schematic diagram illustrating the process of optimizing an isolated forest using a swarm intelligence algorithm based on a cultural framework, as described in this invention.

[0067] Figure 4 This is a schematic diagram of the original isolated forest ROC curve of this invention;

[0068] Figure 5 This is a schematic diagram of the ROC curve of the isolated forest after optimization by the swarm intelligence optimization algorithm according to the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] According to the present invention, a network anomaly detection method based on swarm intelligence algorithm and isolated forest includes: reading a dataset and converting the dataset into a format suitable for constructing an isolated forest, and randomly dividing it into a training set and a test set; constructing an isolated forest model for network fault detection based on the training set; fusing and optimizing the artificial swarm algorithm and the predator algorithm to generate a swarm intelligence optimization algorithm under a cultural framework; and using the swarm intelligence optimization algorithm under the cultural framework to optimize the isolated forest model, thereby finding isolated trees with good detection capabilities to participate in the integration, improving the accuracy of network anomaly detection, and improving the running speed of the isolated forest by deleting isolated trees with poor detection capabilities.

[0071] In this embodiment, a swarm intelligence algorithm based on a cultural framework and a network anomaly detection method based on isolated forests are used, such as Figure 3 As shown, the steps are as follows:

[0072] S1: Training the model

[0073] KDD99 was used as the dataset for network anomaly detection. The input dataset was read and converted into a format suitable for constructing an isolation forest. The dataset was then divided into training and test sets in a 7:3 ratio. The isolation forest model selects all isolated trees to participate in the ensemble. The base classifier in the isolation forest is the isolation tree. During the construction of the isolation tree, a feature w is randomly selected from the dataset D = {d1, d2, ..., dn}. Then, a cutoff value p is randomly selected, with the value ranging between the upper and lower bounds of the selected feature. The data is then divided into left and right parts by the cutoff value. If di(w) > p, the data di(w) is assigned to the right subtree; otherwise, it is assigned to the left subtree. This process is recursively repeated until each data point is completely separated from the other data or the maximum height of the tree is reached. Since outliers are much fewer than normal values ​​and their values ​​differ significantly from normal values ​​in one or more dimensions, and outliers are often closer to the root node of the tree, separating an outlier from the rest of the data is much easier than separating a normal value from the rest of the data. This allows for the separation of outliers from normal values.

[0074] Traverse the isolation tree from the root node to the outer node, and the length of the edges traversed is the path length, denoted as h(x). The path length of instance x is shown in formula (1):

[0075]

[0076] Where H(i) = Ln(i) + γ is the harmonic number, α is the number of leaf nodes, and c(α) is the average value of h(x) when α is determined, used to standardize h(x). Any anomaly detection method requires anomaly scores. The anomaly score of instance x is determined based on the path length of x in all isolation trees, as shown in formula (2):

[0077]

[0078] Where E(h(x)) is the average value of h(x) on the isolation tree set, when:

[0079] E(h(x))→c(n), s→0.5, the sample may not have obvious outliers;

[0080] E(h(x))→0, s→1, can be considered outliers;

[0081] E(h(x))→n-1, n≥1, s→0, can be considered a normal point.

[0082] S2: Algorithm Fusion

[0083] The artificial fish swarm algorithm and the predator algorithm are integrated and optimized to improve the traditional swarm intelligence optimization algorithm, which suffers from slow convergence rate and is prone to getting trapped in local optima.

[0084] The steps to construct an artificial fish swarm algorithm are as follows:

[0085] Initialization settings include population size N, initial position of each artificial fish, visual field of the artificial fish, breakstep, crowding factor δ, and number of repetitions.

[0086] Notice board assignment: Calculate the fitness value of each individual fish in the initial fish swarm, and assign the optimal artificial fish state and its fitness value to the notice board.

[0087] Behavior selection: Each individual is evaluated and its behavior is selected, including foraging, swarming, tailing, and random behavior. According to the behavior selection criteria of the artificial fish swarm algorithm, tailing and swarming behaviors are performed. The default behavior is foraging. If the current best value has not changed after tailing and swarming behaviors, random behavior is performed, and the step size is incremented by one.

[0088] Perform the behavior of artificial fish to update and generate new fish schools.

[0089] Foraging behavior: This mimics the behavior of fish hunting for food in the water. This function sets the vision to the artificial fish knowing the amount of food in a certain place in the water and choosing the direction of swimming accordingly. It is an activity of moving closer to food.

[0090] This behavior first establishes the current state X of an artificial fish. i And within the field of vision of the artificial fish, another state X is randomly selected. j If X j The objective function returns a value better than the current state X under the given condition. i The value of X is then sent to X. j If the state moves one step further, then X... j The objective function returns a value worse than the current state X under the given condition. i If the value is not specified, then a new state is randomly selected and the above steps are repeated. If the artificial fish has reselected a state a maximum number of times in this state and still cannot meet the update condition, then the artificial fish will randomly select a state and move one step. The foraging behavior can be simulated by formula (3):

[0091] X j =X i +Visual*Rand();

[0092]

[0093] Swarming behavior: To hunt in groups and avoid predators, artificial fish usually choose to gather with other artificial fish, forming large or small schools. In these schools, the artificial fish observes the total number of all artificial fish in the current school, calculates the center position of the school, and compares the objective function value obtained at the center position with the objective function value obtained at the current position of the artificial fish. If the objective function value at the center position is better than the objective function value at the current position of the artificial fish, and the crowding at the center position has not reached the threshold, then the artificial fish moves one step from its current position to the center position. Otherwise, it will switch to Prey foraging behavior. Swarming behavior can be simulated by formula (4):

[0094]

[0095] Tail-chasing behavior: This behavior of artificial fish means that if an artificial fish in this school finds food while moving in this area of ​​water, other artificial fish nearby will know the location of the food and will move towards the location of the artificial fish that found the food. Then, artificial fish in more distant locations will also move towards this location.

[0096] The artificial fish will search for the best location among nearby artificial fish. If the value obtained by the objective function under the condition of that best location is better than the value obtained by the objective function under the condition of the current location of the artificial fish, and the crowding of the best location has not reached the threshold, then it means that this location meets the optimization requirements. At this time, the artificial fish will move one step to the optimal location. Otherwise, the algorithm will switch to Prey() foraging behavior. The tail-chasing behavior can be simulated by formula (5):

[0097]

[0098] Bulletin Board Update: Evaluate all individuals. If an individual's fitness value is better than the bulletin board's fitness value, update the bulletin board to that fitness value. Evaluate the overall state of each individual in the current environment based on the position of each fish in the current artificial fish swarm. Compare the overall evaluation obtained by each individual with the best evaluation recorded on the bulletin board. If there is an individual in this iteration that is more in line with the goal than the best evaluation recorded on the bulletin board, then change the best record on the bulletin board to the value represented by that individual.

[0099] The algorithm terminates when the fitness value on the bulletin board reaches the global optimal solution or the upper limit of the number of iterations; otherwise, proceed to step S203.

[0100] The predator algorithm includes the following steps:

[0101] The predator population randomly initializes position x within the search range. i

[0102] x i =rand(1,d)*(ub-lb)+lb, 0<i≤N (6)

[0103] Where x i Here, lb is the predator's position, ub is the minimum (lower bound) of the problem variable, d is the maximum (upper bound) of the problem variable, and d is the number of problem variables (dimension). To expand the search range for solutions, the idea of ​​inverse competition is introduced: Let x∈[lb, ub] be a real number, then its corresponding global inverse number is defined as: The above operations yield 2N initial solutions.

[0104] The following formula is used to search for predators:

[0105] x ij (t+1)=x ij (t)+0.5[(2CZP pos(j) -x ij (t))+(2(1-C)Zμ(j)-x ij (t))](8)

[0106] Where x(t) is the current predator position, x(t+1) is the predator's next iteration position, and P pos Z is the location of the prey, μ is the average value across all locations, and Z is calculated by the following formula.

[0107]

[0108] in, and Let P be a random vector within the range [0,1]. The index value, R2 is a random number in [0,1], and IDX is a vector that satisfies the condition P==0. The index value, C is a balance parameter between exploration and development, whose value decreases from 1 to 0.02 during the iteration process, calculated as follows:

[0109]

[0110] Where it is the current iteration number, and MaxIt is the maximum iteration number. Calculate the prey's position P. pos In order to first calculate the average value (μ) of all locations according to Equation (11), and then calculate the distance between each search individual and the average location.

[0111]

[0112] Calculate the Euclidean distance using equation (12):

[0113]

[0114] According to the hunting scenario, when a predator captures prey, the prey dies, and the predator moves to a new prey location the next time. Therefore, a new diminishing returns mechanism is introduced, as shown in equation (13):

[0115] kbest=round(C×2N) (13)

[0116] Where N is the number of search agents, and due to the use of reverse competition theory, it is 2N in formula (13). The location of the prey is calculated as in formula (14):

[0117]

[0118] At the start of the algorithm, the value of kbest is equal to 2N. The last individual searched that is furthest from the average position (μ) of the searched individuals is selected as prey and captured by the predator. It is assumed that the optimal safe position is the optimal global position, as this would give the prey a better chance of survival, and the predator might choose another prey. Equation (15) is used to update the prey position:

[0119] x ij (t+1)=T pos(j) +R ij (t)(T pos(j) -x ij (t)) (15)

[0120] Where x(t) is the current prey position, and x(t+1) is the prey's next iteration position; T pos It is the globally optimal position; R ij To select predators and prey, a random number in the range [0,1] is used. Combining equations (8), (9), and (15), equation (16) is proposed:

[0121]

[0122] Where r4 is a random number in the range [0,1], and α is an adjustment parameter.

[0123] The artificial fish swarm algorithm and the predator algorithm are integrated and optimized within the cultural algorithm framework:

[0124] Swarm intelligence optimization algorithms, as intelligent algorithms that simulate the collective behavior of social animals, each have their own characteristics and advantages, as well as their own limitations and shortcomings. Using a single swarm intelligence algorithm is rarely sufficient to achieve ideal results. Therefore, by fully utilizing the strengths of different algorithms and compensating for their weaknesses, we study hybrid algorithms to create a comprehensive algorithm that complements their respective advantages. The artificial fish swarm algorithm possesses powerful global search capabilities, while the predator algorithm has strong local optimization capabilities. To overcome the shortcomings of single algorithms, the culture algorithm combines them into a belief space and a group space, forming a novel multi-swarm intelligence optimization algorithm.

[0125] like Figure 1 As shown, individuals in the population space evolve according to the predator algorithm mechanism, while individuals in the faith space evolve according to the artificial fish swarm algorithm mechanism. The role of the faith space is to guide population optimization. The two spaces are independent yet interconnected, linked together by a communication protocol composed of acceptance and influence functions. The lower population space periodically contributes elite individuals to the upper faith space. After evolution, the upper faith space sends some of the better individuals into the population space to replace the worse individuals, thus forming a collaborative mechanism of "dual evolution and dual promotion".

[0126] The role of the acceptance function is to transfer individual experience to the belief space. Its main function is to select dominant individuals from the current population space, providing a foundation for the evolution of the belief space. The belief space is updated by selecting superior individuals from the current population through the acceptance operation. Each time AcceptStep is run, the worst individual in the belief space is replaced with the current global optimum in the population space. The calculation formula for AcceptStep is as follows:

[0127]

[0128] Where is the fix rounding function, MAXIt is the pre-set maximum number of generations of population evolution, It is the current generation of population evolution, BaseNum and DevNum are freely set constants; when It%AcceptStep=0, the worst individual in the belief space is replaced by the current global optimal value in the population space.

[0129] The influence function modifies the behavioral rules of individuals in the population space to achieve higher evolutionary efficiency, thereby improving population diversity and algorithm efficiency. Through influence operations, superior individuals from the evolved belief space are transferred into the population space to replace inferior individuals.

[0130]

[0131] Where is the fix rounding function, MAXIt is the pre-set maximum number of generations of population evolution, It is the current generation of population evolution, BaseNum and DevNum are freely set constants; when It%AffecttStep=0, the worst individual in the population space is replaced by the current global optimal value in the belief space.

[0132] S3: The traditional isolated forest model is optimized using an improved swarm intelligence optimization algorithm. The detection rate is used as the fitness function. The optimal solution corresponding to the swarm intelligence optimization algorithm is the isolated tree with high dissimilarity and high detection rate. The isolated trees with high dissimilarity and high detection rate are selected to participate in the integration, which improves the accuracy of network anomaly detection. At the same time, the running speed of the isolated forest is improved by deleting isolated trees with low detection rate.

[0133] like Figure 2As shown, firstly, 100 training subsets containing 2048 samples each are randomly sampled from the training set using the Bootstrap random sampling method. Isolation trees are then trained on each training subset, resulting in 100 isolation trees. Next, the traditional Isolation Forest algorithm is optimized using an artificial fish swarm and predator algorithm within a cultural framework to identify superior isolation trees for ensemble integration. Using the anomaly detection rate as the fitness function, this method improves the generalization performance of the traditional Isolation Forest model by eliminating poorly performing isolation trees, thereby increasing the fault detection rate.

[0134] To verify the effectiveness of the techniques, KDD99 was used as the dataset for network anomaly detection. KDD99 is a dataset used to detect abnormal connections from normal connections. In the experiments, three network anomaly detection methods were tested on the dataset: the original Isolation Forest algorithm, the LOF algorithm, and the Isolation Forest algorithm based on swarm intelligence optimization. The LOF algorithm is based on density analysis, detecting anomalies through local data density. Because anomaly detection in this paper is a binary classification problem, AUC values ​​and ROC curves were used to evaluate the algorithm's performance. In the ROC curve, the x-axis represents recall, and the y-axis represents precision. In the confusion matrix, TP represents correctly identified positive examples, TN represents correctly identified negative examples, FP represents incorrectly identified positive examples, and FN represents incorrectly identified negative examples.

[0135] Precision refers to a proportion that represents the percentage of samples that were predicted to be true and also proved to be true out of the total number of samples that were predicted to be true. It can be obtained by the following formula (19):

[0136]

[0137] Recall is the proportion of eligible samples that are found in a batch of samples, and it can be obtained by the following formula (20):

[0138]

[0139] The x-axis of the AUC curve is FPR, which can be obtained by formula (21), and the y-axis is TPR, which can be obtained by formula (22). FPR represents the probability of misclassifying negative data as positive data, and TPR represents the probability of correctly classifying positive data. AUC can be obtained by formula (23), which is the area under the ROC curve.

[0140]

[0141]

[0142]

[0143] Where M is the number of positive samples and N is the number of negative samples.

[0144] The KDD99 dataset was verified using the LOF algorithm, the original isolation forest algorithm, and the isolation forest based on the swarm intelligence optimization algorithm. The results are shown in Table 1.

[0145] Table 1 Comparison of algorithm performance

[0146] algorithm Accuracy AUC Modeling time Predicted duration LOF algorithm 88％ 0.67 46min 1min Original Isolation Forest Algorithm 95％ 0.80 2min 4s Improved Isolation Forest Algorithm 96％ 0.85 11min 2s

[0147] As can be seen from Table 1, the performance of the improved isolation forest algorithm is better than that of the LOF algorithm and the original isolation forest algorithm. Among them, the accuracy is higher, and the larger the value of AUC indicates the higher authenticity of the detection method, and the faster the convergence speed.

[0148] Figure 4 It is a schematic diagram of the ROC curve of the original isolation forest. Figure 5 It is a schematic diagram of the ROC curve of the isolation forest optimized by the swarm intelligence optimization algorithm; from Figure 4 and Figure 5 it can be seen that the area under the ROC curve of the isolation forest optimized by the swarm intelligence optimization algorithm is larger than that of the original isolation forest ROC curve. The larger the area, the better the classification effect. Therefore, the optimized isolation forest algorithm is better, the authenticity of the detection method is higher, and the convergence speed is faster.

[0149] The improved swarm intelligence optimization algorithm is used to optimize the traditional isolation forest model, and isolation trees with high quality are selected to participate in the integration to improve the accuracy of network anomaly detection. At the same time, by deleting some isolation trees with poor quality, the running speed of the isolation forest is improved.

[0150] The above are only the preferred embodiments of the present invention, which are illustrative rather than restrictive to the present invention. Those skilled in the art understand that many changes, modifications, and even equivalents can be made within the spirit and scope defined by the claims of the present invention, but all will fall within the protection scope of the present invention.

Claims

1. A network anomaly detection method based on swarm intelligence algorithm and isolated forest, characterized in that, Includes the following steps: S1: Training the model The dataset is randomly divided into training and testing sets, and an isolated forest model for network fault detection is constructed based on the training set. S2: Swarm Intelligence Optimization Algorithm A novel swarm intelligence optimization algorithm is obtained by fusing and optimizing the artificial fish swarm algorithm and the predator algorithm based on the culture algorithm: The cultural algorithm includes a population space and a belief space. Individuals in the population space evolve according to the predator algorithm mechanism, while individuals in the belief space evolve according to the artificial fish swarm algorithm mechanism. The population space and the belief space are linked together through a communication protocol composed of acceptance functions and influence functions; The population space contributes elite individuals to the faith space, where the elite individuals refer to the optimal solutions generated during the evolution of the predator algorithm in the population space. After evolution, the faith space sends superior individuals into the population space to replace inferior individuals. The superior individuals are those with high recognition accuracy generated during the evolution of the artificial fish swarm algorithm in the faith space, while the inferior individuals are those with low recognition accuracy generated during the evolution of the predator algorithm in the population space. S3: Model Optimization The isolated forest model is optimized using a swarm intelligence optimization algorithm, with the anomaly detection rate used as the fitness function. The swarm intelligence optimization algorithm selects high-accuracy isolated trees from the isolated forest model to participate in the ensemble, and removes isolated trees with low accuracy; finally, the algorithm's performance is tested on the test set.

2. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 1, characterized in that, S1 specifically includes the following steps: S11: First, collect the dataset for network anomaly detection, and divide the dataset into training set and test set; randomly select n sample data points from the training set as a sub-sample set, construct an isolation tree, and put the sub-sample set into the root node of the isolation tree; A feature w is randomly selected from the subsample set, and then a cutoff value p is randomly selected. The cutoff value p is between the upper and lower values ​​of the selected feature w. The sample data is divided into two parts, left and right, by the splitting value p. If di(w) > p, the data di(w) is divided into the right subtree, and vice versa. Recursively construct new child nodes until only one data item in a child node cannot be further cut, at which point the isolation tree has reached the initially defined limit height. S12: Construct n isolated trees as described above to form an isolated forest model; S13: Select test data x in the test set and substitute it into each isolated tree in the isolated forest model; x represents the test data. Calculate the path length h(x) that falls on each isolated tree, and calculate the average value E(h(x)) of all h(x). The path length c of the test data x can be obtained using the following formula ( ), ,in Harmonic number, c is the number of leaf nodes. ) for when When determined The average value; Calculate the outlier score of test data x using the following formula. : ； Based on abnormal scores The value of the value is used to distinguish between outliers and normal points, thus separating the abnormal data from the normal data in the test set.

3. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 1, characterized in that, The cultural algorithm in S2 for the fusion and optimization of the predator algorithm and the artificial fish swarm algorithm specifically includes the following steps: The belief space is updated by selecting the better individuals in the current population space through the accept operation. Each time AcceptStep is run, the worst individual in the belief space is replaced with the current global best value in the population space. The calculation formula of AcceptStep is as follows. ； in: To receive the number of operations, Integer function, The maximum number of generations that a population can evolve is pre-defined. For the current generation of population evolution, and Set constants for freedom; When satisfied When the worst individual in the belief space is replaced by the current global optimum in the population space; By influencing the behavior, superior individuals from the evolved belief space are sent into the population space to replace inferior individuals. ； in: To affect the number of operations; when the following conditions are met When the worst individual in the population space is replaced by the globally optimal value in the belief space, the worst individual in the population space is replaced by the globally optimal value in the belief space.

4. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 3, characterized in that, The artificial fish swarm algorithm includes the following steps: S201: Initialization settings, including population size, initial position of each artificial fish, field of vision of the artificial fish, step size, crowding factor, and number of repetitions; S202: Bulletin board assignment: Calculate the fitness value of each individual in the initial fish swarm, and assign the optimal artificial fish state and its fitness value to the bulletin board; S203: Behavior selection: Evaluate each individual and select the behavior to be performed, including foraging, herding, tail chasing and random behavior; S204: Perform the behavior of artificial fish and update to generate a new fish swarm; S205: Bulletin Board Update: Evaluate all individuals. If an individual's fitness value is better than the bulletin board value, update the bulletin board value to that fitness value. S206: The algorithm ends when the fitness value on the bulletin board reaches the global optimal solution or the upper limit of the number of iterations; otherwise, proceed to step S203.

5. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 3, characterized in that, The predator algorithm includes the following steps: S211: Initialization phase, the predator randomly initializes its position within the search area. ; S212: Predator search phase, find the predator's next iteration position. .

6. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 5, characterized in that, The initialization position in S211 Calculated using the following formula , ; in It is the initial position of the i-th predator. It is the minimum value of the problem variable. It is the maximum value of the problem variable. This refers to the number of problem variables, where the problem variables refer to the solution space vector; To expand the search range for solutions, the idea of ​​reverse competition is introduced: Let... ∈[ , If ] is a real number, then its corresponding global reverse number is defined as: Through the above operations, 2N initial solutions are obtained.

7. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 5, characterized in that, S212 includes the following steps: S2121: Find the predator's next iteration position. , ,in This is the current predator's position. It is the location of the prey. Z is the average of the predator's position and the prey's position, Z is the adaptive parameter, and C is the balance parameter. S2122: Locate the prey | is sorted , where kbest=round(C N), where N is the number of search agents and kbest is the location of the prey search agent; S2123: Request the updated location of the prey , ;in, This is the current location of the prey. It is the next iteration position of the prey. It is the globally optimal position; It is a random number in the range [0,1].

8. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 7, characterized in that, The adaptive parameter Z in S2121 is calculated using the following formula: Z= + ， ； and Let P be a random vector within the range [0,1]. index value, It is a random number within [0,1], and IDX is a number that satisfies the condition. vector The index value.

9. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 7, characterized in that, The equilibrium parameter C in S2121 is calculated using the following formula. Where it is the current iteration number; the average position value. Calculated using the following formula .

10. The network anomaly detection method based on swarm intelligence algorithm and isolated forest as described in claim 9, characterized in that, Each predator and its position average in S2122 distance Calculated using the following formula .

Images