A multi-level network security situation assessment method

Abstract

The present invention relates to the fields of machine learning and network security, and discloses a multi-level network security situation assessment method, comprising the following steps: pre-normalizing, attribute reduction, data cleaning, and missing value processing for data of different attack risk sub-indicators; evaluating the attack risk sub-indicators using the LightGBM algorithm; combining the attack risk sub-indicators with weights using the LightBGM algorithm to calculate the attack risk index; and iteratively optimizing the attack risk sub-indicator weights using a simulated annealing algorithm. The multi-level network security situation assessment method can address the large-dimensional and wide-ranging data of different attack risk sub-indicators, which require normalization, attribute reduction, data cleaning, and missing value processing. The sub-feature data of each attack risk sub-indicator is input into its respective LightGBM model in parallel through multiple processes for training, thereby obtaining a score for the attack risk sub-indicator.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and network security, and in particular to a multi-level network security situation assessment method. Background Art

[0002] Cyberspace sovereignty means sovereignty over the information and communication technology activities that occur in this space, the information and communication technology platforms themselves, and their data.

[0003] As the scale and complexity of networks continue to grow, network attack technologies continue to innovate, and new attack tools emerge in large numbers. Traditional network security technologies seem to be unable to cope with the situation, network intrusions are inevitable, and network security issues are becoming more and more serious.

[0004] Situational awareness risk assessment plays a crucial role in studying the impact of cyberattacks on networks. It is the focus and key link in the entire process. A network security situation assessment involves aggregating, filtering, and correlating security events generated by devices and then establishing appropriate mathematical models based on security indicators to assess the overall security threat level faced by the network system. This allows the network to be analyzed at what stage it is currently under attack and provide a comprehensive understanding of the overall network security status.

[0005] Situational awareness and risk assessments can identify network security risks and threats early. A thorough assessment of the scope and severity of these risks and threats can help managers understand the current network security status, enabling them to take containment and prevention measures before they occur, protecting the system from attack and damage. Only by assessing the situation can we clearly understand the network's security status, understand the overall network security situation, and provide a basis for further situation prediction.

[0006] Due to the importance of network security situation awareness assessment for maintaining cyberspace security, research and application in this field are becoming increasingly active. Currently, there are many network security situation assessment methods based on graph structures. A multi-step attack security situation awareness assessment method based on a Bayesian network uses Bayesian network posterior inference and cumulative probability to calculate multi-step attack risks, and adopts a hierarchical quantitative assessment method to quantitatively assess the security situation of the host and the entire network. This method can solve the problem of lack of correlation analysis in the network security situation assessment process and take monitoring events into account in risk assessment. There is also a security situation assessment method based on attack graph status sorting, but this method is not accurate in attack graph status sorting, which greatly affects the accuracy of network security situation assessment. There is also a network security situation awareness assessment method based on set pair analysis. This method can evaluate from multiple perspectives such as uniformity, difference, and opposition measurement, avoiding the limitations of using a single standard, but lacks scientific basis and recognized methods for constructing the same-different-anti-connection degree. There is also a method based on deep learning feature extraction that combines rough set analysis and deeply abstracted pattern information to provide an assessment strategy. However, rough set theory has the drawbacks of high computational complexity and non-real-time performance. There is also a method that uses BP neural network for network security situation assessment. However, as the size of the data set increases, the hidden nodes and data dimensions of the neural network increase exponentially, which makes neural networks ineffective for large-scale and complex network security situation assessment. The above mentioned network security situation assessment methods are not perfect and are all limited by the limitations of their model algorithms.

[0007] The Bayesian network-based multi-step attack security situation awareness assessment method uses frequent mining to extract multi-step attack patterns and construct an attack graph, thereby generating a network security situation assessment model. However, this model ignores the fact that not all nodes in a Bayesian network are relatively independent, which results in significant errors in the overall network security situation assessment. However, when using attack graph state ranking for network security situation assessment, inaccurate ranking can affect the assessment results. While using set pair analysis (SETA) based on mathematical models can achieve good results across multiple metrics, it lacks a scientific basis. Network security situation assessment based on rough set theory using deep learning can be inefficient and resource-intensive when faced with massive amounts of data. However, methods using BP neural networks for network security situation assessment suffer from exponential growth in the number of hidden nodes and data dimensions as the dataset size increases, resulting in poor performance when using neural networks for large-scale, complex network security situation assessment. Therefore, we propose a multi-level network security situation assessment method. Summary of the Invention

[0008] (1) Technical problems solved

[0009] In view of the deficiencies of the existing technology, the present invention provides a multi-level network security situation assessment method to solve the above problems.

[0010] (2) Technical solution

[0011] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a multi-level network security situation assessment method, comprising the following steps:

[0012] Step 1: Normalize, reduce attributes, cleanse, and handle missing values ​​for each attack risk indicator.

[0013] Step 2: Use the LightGBM algorithm to evaluate the attack risk indicators;

[0014] Step 3: Use the LightBGM algorithm to combine the attack risk index and weight to calculate the attack risk index;

[0015] Step 4: Iteratively optimize the attack risk index weights using the simulated annealing algorithm.

[0016] Preferably, the specific content of the second step is:

[0017] The number of leaves per tree, tree depth, the minimum number of samples on a leaf, and the minimum sample set parameters required to establish a leaf are determined based on the size of the data set. The number of leaves per tree must be strictly less than 2. max_depth ;

[0018] The tree depth should not be set too deep. The minimum sample set parameter required to build a leaf should be as large as possible according to the data set.

[0019] Perform faster bagging of results using the sample data random sampling ratio parameter, which determines the feature subset used in each iteration.

[0020] Preferably, the specific content of the third step is as follows:

[0021] S1: The scores of each attack risk index calculated in parallel from multiple LightGBM models are saved in a matrix;

[0022] S2: Combine the attack risk score obtained by the LightGBM model with its randomly initialized weights to obtain the attack risk index. Compare the calculated attack risk index with the actual attack risk index to calculate the mean square error (MSE), which is used as the objective function.

[0023] S3: The simulated annealing algorithm then makes a judgment based on the current attack risk index and the previous attack risk index, and randomly perturbs the weight corresponding to each attack risk index in a positive or negative direction, that is, adding or subtracting a random number from the weight, and the random number is limited to a range;

[0024] S4: Multiply the updated weights by the corresponding attack risk sub-index and sum them. Compare the result of the previous step with the actual value and solve the mean square error. If the currently calculated objective function is better than the previous objective function, then accept the current attack risk sub-index weights and objective function as the most weighted and optimal objective function. Otherwise, selectively accept the new attack risk sub-index weights and objective function according to the Monte Carlo judgment criterion. Repeat the above optimization process until the temperature drops to the pre-set minimum temperature, thereby obtaining the most accurate situation assessment model.

[0025] Preferably, the specific steps in the fourth step are as follows:

[0026] S1: Initialize the weight ω corresponding to the attack risk index i , combined with the attack risk index to obtain the attack risk index, the calculated attack risk index is compared with the actual value and the mean square error is calculated as the initialization objective function;

[0027] S2: According to the objective function, the weight of the attack risk index is perturbed in a positive or negative direction. The perturbation result is combined with the attack risk index calculated by the LightGBM algorithm to calculate the attack risk index of the current iteration process. The calculated value is compared with the actual attack risk index, and the mean square error of the two is calculated as the objective function of the current iteration process.

[0028] S3: Determine whether the objective function of the current iteration process is better than the objective function of the previous iteration process. If it is better than the solution of the previous iteration process, the weight of the attack risk index and the objective function of the current iteration process are accepted as the optimal solution. Otherwise, the new solution is selectively accepted according to the Metropoils judgment criterion. The Metropoils judgment criterion formula is as follows:

[0029]

[0030] T is the initial temperature set by the simulated annealing algorithm, and k is the speed at which the temperature decreases;

[0031] S4: After the Metropoils judgment criterion is applied, it is determined whether the number of iterations of the current loop has been reached. If so, the current loop is exited; otherwise, the above process is continued.

[0032] S5: Determine whether the preset minimum temperature is reached. If so, exit the entire loop and return to the global optimal solution. Otherwise, slowly lower the temperature, control the temperature drop rate according to the k value, and reset the number of iterations of the upper loop.

[0033] (3) Beneficial effects

[0034] Compared with the existing technology, the present invention provides a multi-level network security situation assessment method with the following beneficial effects:

[0035] 1. This multi-level network security situation assessment method can address the large and wide-ranging data dimensions of various attack risk indicators, which require normalization, attribute reduction, data cleaning, and missing value processing. The sub-feature data of each attack risk indicator is fed into its own LightGBM model through multiple parallel processes for training, thereby obtaining the attack risk indicator score.

[0036] 2. This multi-level network security situation assessment method uses the LightGBM algorithm to train datasets in parallel, resulting in high speed, low memory consumption, and the ability to screen the most accurate attack risk indicators during the network security situation assessment process. This method uses a simulated annealing algorithm to optimize the weights corresponding to the attack risk indicators, combining the attack risk indicators with their weights to derive an attack risk index. By comparing the calculated attack risk index with the actual attack risk index, the attack risk indicator weights are continuously optimized and updated to determine the optimal situation assessment model.

[0037] 3. This multi-level network security situation assessment method first normalizes, attributes, cleans, and handles missing values ​​for the sub-feature data of each attack risk sub-indicator. It then evaluates the attack risk sub-indicators based on the LightGBM algorithm. The attack risk index for the current iteration is then derived by combining randomly initialized weights with the calculated attack risk sub-indicators. The calculated value is compared with the true attack risk index, and their mean square error is calculated as the initial objective function. The objective functions of the current iteration and the previous iteration are then compared using a simulated annealing algorithm. If the current iteration is better than the previous iteration's objective function, the weights and objective function of the attack risk sub-indicators for the current iteration are accepted. Otherwise, the new solution is selectively accepted using the Metropoils judgment criterion. This process is repeated until the entire simulated annealing algorithm is exited, and the global optimal solution is returned, thereby calculating the most accurate situation assessment model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart for the implementation of a multi-level network security situation assessment method based on simulated annealing algorithm and LightGBM algorithm;

[0039] Figure 2 This is a flowchart for iteratively updating the attack risk sub-index weights in LightGBM using the simulated annealing algorithm;

[0040] Figure 3 It is a multi-level attack risk assessment factor. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] See also Figure 1-3 , a multi-level network security situation assessment method, comprising the following steps:

[0043] Step 1: Process the sub-feature data of the attack risk index;

[0044] The data dimensions of different attack risk indicators are large and wide-ranging, and they need to be normalized, attribute-reduced, data cleaned, and missing value processed in advance.

[0045] Step 2: LightGBM algorithm evaluates attack risk indicators;

[0046] First, the processed attack risk sub-indicator sub-feature data is input into each LightGBM model in parallel through multiple processes for training. The attack risk sub-indicator is scored using the LightGBM algorithm. The score of the attack risk sub-indicator is calculated by finding the optimal split point on different feature sets of different processes.

[0047] The following optimizations were made to the LightGBM algorithm: first, a relatively high learning rate was selected to speed up the convergence speed, and then the number of boosting iterations was determined. In the subsequent parameter adjustment process, the appropriate learning rate and number of iterations were selected according to the data set. In order to improve the accuracy of the LightGBM model for attack risk indicators. Since the LightGBM algorithm is a decision tree algorithm based on Histogram (histogram), the decision tree grows according to the leaf-wise (leaf growth) algorithm. The num_leaves (number of leaves per tree), max_depth (tree depth), min_data_in_leaf (the minimum number of samples on a leaf), and min_child_samples (the minimum set of samples required to establish a leaf) parameters are determined according to the size of the data set. Num_leaves needs to be strictly less than 2 max_degth Setting this parameter too high will result in a decision tree that is too deep, which will lead to overfitting as accuracy improves. Therefore, it is necessary to set the parameter appropriately based on the size of the dataset. Similarly, setting max_depth too high will cause overfitting. min_data_in_leaf is affected by num_leaves and the dataset. Setting this parameter appropriately based on the training set can avoid overly deep decision trees. The min_child_samples parameter should be set as high as possible based on the dataset to reduce overfitting.

[0048] Use the bagging_fraction (random sampling ratio of sample data) parameter to perform faster result bagging, and feature_fraction (random sampling ratio of feature columns) to determine the feature subset used in each iteration.

[0049] Step 3: The LightBGM algorithm combines the attack risk index and weight to calculate the attack risk index;

[0050] The scores of each attack risk index are calculated in parallel from multiple LightGBM models, and all the scores of each attack risk index are saved in a matrix. The attack risk index obtained by the LightGBM model is first combined with its randomly initialized weights to obtain the attack risk index. The calculated attack risk index is compared with the actual attack risk index to calculate the mean square error and the mean square error is used as the objective function (the objective function is the core judgment basis for the simulated annealing algorithm to update and optimize the attack risk index weights). The simulated annealing algorithm then makes a judgment based on the current attack risk index and the previous attack risk index. It randomly perturbs the weights corresponding to each attack risk index in a positive or negative direction (i.e., increases or decreases the weight by a random number [the random number is limited to a range]). The updated weights are multiplied by the corresponding attack risk index and the sum is calculated. The result of the previous step is compared with the actual value and the mean square error is calculated. If the currently calculated objective function is better than the previous objective function, the current attack risk index weights and objective function are accepted as the most weighted and optimal objective function. Otherwise, the new attack risk index weights and objective function are selectively accepted according to the Metropoulos (Monte Carlo) judgment criterion. The above optimization process is repeated until the simulated annealing algorithm is exited (the temperature drops to the preset minimum temperature), thereby obtaining the most accurate situation assessment model.

[0051] Step 4: Iteratively optimize the attack risk sub-index weights based on the simulated annealing algorithm;

[0052] Initialize the weight ω corresponding to the attack risk index i , combined with the attack risk index to obtain the attack risk index. Compare the calculated attack risk index with the actual value and calculate the mean square error sum as the initial objective function. According to the objective function, perform positive and negative perturbations on the weight of the attack risk index, combine the perturbation result with the attack risk index calculated by the LightGBM algorithm to calculate the attack risk index of the current iteration process, compare the calculated value with the actual attack risk index, and calculate the mean square error sum of the two as the objective function of the current iteration process. Determine whether the objective function of the current iteration process is better than the objective function of the previous iteration process. If it is better than the solution of the previous iteration process, accept the weight of the attack risk index and the objective function of the current iteration process as the optimal solution. Otherwise, the new solution will be selectively accepted through the Metropoils judgment criterion. The formula of the Metropoils judgment criterion is as follows:

[0053]

[0054] In the above formula, T is the initial temperature set by the simulated annealing algorithm, and k is the rate of temperature drop. After applying the Metropoils criterion, the algorithm determines whether the current loop iteration limit has been reached (the length of the Markov chain requires multiple debugging sessions to determine the optimal number of iterations). If so, the algorithm exits the current loop; otherwise, the above process continues. The algorithm then determines whether the preset minimum temperature has been reached. If so, the algorithm exits the entire loop and returns to the global optimal solution. Otherwise, the algorithm slowly decreases the temperature (controlling the rate of temperature drop based on the value of k), resetting the iteration limit of the previous loop.

[0055] The weights of each attack risk sub-indicator are iteratively optimized through the simulated annealing algorithm, so that the weights are combined with the attack risk sub-indicators. The calculated attack risk index greatly improves the accuracy of network security situation assessment.

[0056] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A multi-level network security situation assessment method, characterized in that: The following steps are involved: Step 1: Normalize, reduce attributes, cleanse, and handle missing values ​​for each attack risk indicator. Step 2: Use the LightGBM algorithm to evaluate the attack risk indicators; Step 3: Use the LightGBM algorithm to combine the attack risk index and weight to calculate the attack risk index; Step 4: Iteratively optimize the attack risk index weights using a simulated annealing algorithm; The specific steps in the fourth step are as follows: S1: Initialize the weights corresponding to the attack risk sub-indicators, combine them with the attack risk sub-indicators to derive the attack risk index, compare the calculated attack risk index with the actual value, and calculate the mean square error (MSE) as the initial objective function. S2: According to the objective function, the weight of the attack risk index is perturbed in a positive or negative direction. The perturbation result is combined with the attack risk index calculated by the LightGBM algorithm to calculate the attack risk index of the current iteration process. The calculated value is compared with the actual attack risk index, and the mean square error of the two is calculated as the objective function of the current iteration process. S3: Determine whether the objective function of the current iteration is better than the objective function of the previous iteration. If so, the weight of the attack risk sub-indicator and the objective function of the current iteration are accepted as the optimal solution. Otherwise, the Metropolis judgment criterion is used to selectively accept the new solution. S4: After the Metropolis judgment criteria, it is determined whether the number of iterations of the current loop has been reached. If so, the current loop is exited, otherwise the above process is continued; S5: Determine whether the preset minimum temperature is reached. If so, exit the entire loop and return to the global optimal solution. Otherwise, slowly lower the temperature, control the temperature drop rate according to the k value, and reset the number of iterations of the upper loop.

2. A multi-level network security situation assessment method according to claim 1, characterized in that: The specific contents of the second step are: The number of leaves of each tree, the tree depth, the minimum number of samples on a leaf, and the minimum sample set parameters required to establish a leaf are determined according to the size of the data set. The number of leaves of each tree needs to be strictly less than ; The tree depth should not be set too deep, and the min_child_samples parameter should be as large as possible according to the dataset; Use bagging_fraction to perform faster bagging of results and feature_fraction to determine the subset of features used in each iteration.

Images