Network security threat detection method and system based on artificial intelligence

Abstract

The present invention provides an artificial intelligence-based network security threat detection method and system, which relate to the field of network security technology, including collecting initial network traffic data, and performing preprocessing operations on the initial network traffic data to generate target network traffic data; classifying the target network traffic data to generate training network traffic data and test network traffic data; building an initial threat identification model based on the training network traffic data; evaluating the recognition effect of the initial threat identification model based on the test network traffic data, and optimizing the initial threat identification model according to the recognition effect to obtain a target threat identification model, so that the network traffic data can be analyzed in real time through deep learning and machine learning technologies, and potential network security threats can be identified, and the accuracy and efficiency of threat detection are improved by optimizing the process of data preprocessing, feature selection and model training, while reducing the false alarm rate.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to an artificial intelligence-based network security threat detection method and system. Background Art

[0002] In the current era of digitalization and networking, cybersecurity has become a global focus. With the rapid development of Internet technology, cyber attack methods are becoming increasingly complex, and traditional rule-based cybersecurity defense systems have been unable to cope with evolving threats. These threats include malware, phishing, denial of service attacks, and advanced persistent threats (APTs), which pose a serious threat to personal privacy and corporate data security. Therefore, it is urgent to develop more efficient and intelligent cybersecurity threat detection technologies.

[0003] However, in the existing technology, due to the high dimensionality and unstructured characteristics of network traffic data, traditional network security threat detection methods cannot accurately extract features from massive data to improve the accuracy and efficiency of threat detection. Secondly, the continuous evolution of network attack methods requires the detection system to have strong adaptability and learning ability, but the existing system often performs poorly when facing new attacks. In addition, traditional network security threat detection methods cannot effectively balance the accuracy and false alarm rate of threat detection, and may cause excessive alarms, which in turn leads to waste of resources.

[0004] Therefore, it is necessary to provide an artificial intelligence-based network security threat detection method and system to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an artificial intelligence-based network security threat detection method and system for solving the problem that traditional network security threat detection methods are unable to accurately extract features from high-dimensional and unstructured network traffic data to improve the accuracy and efficiency of threat detection; the continuous evolution of network attack methods requires the detection system to have strong adaptability and learning ability, but the existing system often performs poorly when facing new types of attacks; traditional network security threat detection methods cannot effectively balance the accuracy and false alarm rate of threat detection, and may cause excessive alarms, thereby leading to the problem of waste of resources.

[0006] The network security threat detection method based on artificial intelligence provided by the present invention includes:

[0007] Collecting initial network traffic data, and performing preprocessing operations on the initial network traffic data to generate target network traffic data;

[0008] Classifying the target network traffic data to generate training network traffic data and test network traffic data;

[0009] Building an initial threat identification model based on the training network traffic data, and continuously updating the training network traffic data;

[0010] The recognition effect of the initial threat recognition model is evaluated based on the test network traffic data, and the initial threat recognition model is optimized according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model.

[0011] Preferably, the collecting of initial network traffic data and performing preprocessing operations on the initial network traffic data to generate target network traffic data specifically includes:

[0012] Acquire a network interface, and collect initial network traffic data from the network interface;

[0013] Receiving the initial network traffic data, and performing an unpacking operation on the initial network traffic data to generate first network traffic data;

[0014] Based on a preset filtering rule, invalid information in the first network traffic data is removed to generate second network traffic data;

[0015] The data format of the second network traffic data is unified to generate the target network traffic data.

[0016] Preferably, the constructing of an initial threat identification model based on the training network traffic data specifically includes:

[0017] Normalizing the training network traffic data to obtain input network traffic data;

[0018] The input network traffic data is trained to obtain the initial threat identification model.

[0019] Preferably, the training network traffic data is normalized to obtain input network traffic data, and the normalization formula is used for processing:

[0020]

[0021] In the formula, Represents training network traffic data; Represents input network traffic data; Represents the minimum value of the training network traffic data; Indicates the maximum value of the training network traffic data; Indicates the minimum value of the target zoom range; Indicates the maximum value of the target zoom range.

[0022] Preferably, the training of the input network traffic data to obtain the initial threat identification model specifically includes:

[0023] Setting a neural network structure corresponding to the initial threat identification model, and determining an activation function type, a number of hidden layers, and a number of neurons of the neural network structure;

[0024] Set the maximum number of training times for the neural network structure and minimum total mean square error ;

[0025] Set the current iteration counter of the neural network structure to 0 and initialize the weight coefficients ,in, Indicates the number of layers; Indicates Layer neurons; Indicates Layer neurons;

[0026] For the input network traffic data, perform signal feedforward propagation and calculate the Layer The input signal of a neuron is:

[0027]

[0028] In the formula, Indicates Layer The input signal of each neuron; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer weight coefficient; Indicates Layer neurons; Indicates Layer neurons; Indicates Layer The actual output of each neuron;

[0029] when hour, and ,in, represents the bias term;

[0030] when hour, ,in, represents the activation function; when hour, .

[0031] Preferably, after the input network traffic data performs signal feedforward propagation, the error back propagation begins, which specifically includes:

[0032] Calculate the accumulation of the weight coefficients:

[0033]

[0034] In the formula, Indicates Accumulation of layer weight coefficients; represents the correction proportional coefficient of the weight, ; Indicates Layer The error term of each neuron; Indicates Layer The actual output of each neuron;

[0035] Jordi The layer is the output layer, and we get

[0036]

[0037] In the formula, Indicates Layer The error term of each neuron; Indicates Layer The input signal of each neuron; Indicates that the activation function is The derivative at ; Indicates Layer The expected output of each neuron; Indicates Layer The actual output of each neuron;

[0038] otherwise

[0039]

[0040] In the formula, Indicates Layer The error term of each neuron; Indicates Layer The input signal of each neuron; Indicates that the activation function is The derivative at ; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer The error term of each neuron; Indicates Layer weight coefficient;

[0041] Adjust the weight coefficient corresponding to the neural network structure:

[0042]

[0043] In the formula, Indicates the adjusted Layer weight coefficient; Indicates Layer weight coefficient; Indicates Accumulation of layer weight coefficients.

[0044] Preferably, after adjusting the weight coefficient corresponding to the neural network structure, the number of iterations corresponding to the neural network structure is increased. , until the termination condition is met, the termination condition is:

[0045]

[0046] or

[0047]

[0048] In the formula, Indicates the current iteration number; Indicates the maximum number of training times; represents the total mean square error; Indicates the number of input signals; represents mathematical expectation; Indicates Layer The input signal of each neuron; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer The expected output of each neuron; Indicates Layer The output of a neuron; represents the minimum total mean square error;

[0049] The output of the neural network structure , which is the output of the initial threat identification model, where Indicates the probability of threat occurrence, and its value range is between 0 and 1.

[0050] Preferably, the evaluating the recognition effect of the initial threat recognition model based on the test network traffic data, and optimizing the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model specifically includes:

[0051] Obtaining a confusion matrix corresponding to the test network traffic data, and evaluating the recognition effect of the initial threat recognition model according to the confusion matrix;

[0052] The elements of the confusion matrix include true positive examples, false positive examples, true negative examples and false negative examples, wherein the true positive examples are test network traffic data correctly identified as threats; the false positive examples are test network traffic data incorrectly identified as threats; the true negative examples are test network traffic data correctly identified as normal; and the false negative examples are test network traffic data incorrectly identified as normal.

[0053] Based on the confusion matrix, calculating the performance indicators of the initial threat identification model, wherein the performance indicators include accuracy, precision, recall, F1 score and false positive rate;

[0054] The calculation formula of the accuracy is:

[0055]

[0056] In the formula, Represents the accuracy of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents a true counterexample, i.e., the test network traffic data that is correctly identified as normal; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; represents a false negative example, i.e., test network traffic data that is mistakenly identified as normal;

[0057] The calculation formula of the accuracy is:

[0058]

[0059] In the formula, Represents the accuracy of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat;

[0060] The calculation formula of the recall rate is:

[0061]

[0062] In the formula, represents the recall rate of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents a false negative example, i.e., test network traffic data that is mistakenly identified as normal;

[0063] The calculation formula of the F1 score is:

[0064]

[0065] In the formula, represents the F1 score of the initial threat identification model; Represents the accuracy of the initial threat identification model; represents the recall rate of the initial threat identification model;

[0066] The calculation formula of the false positive rate is:

[0067]

[0068] In the formula, represents the false positive rate of the initial threat identification model; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; represents a true counterexample, i.e., the test network traffic data that is correctly identified as normal;

[0069] The performance indicators are analyzed, and the precision and recall are weighed to evaluate the recognition effect of the initial threat recognition model. That is, a high precision and a low recall rate indicate that the initial threat recognition model rarely misjudges normal traffic as a threat, but sometimes misses a real threat; a low precision and a high recall rate indicate that the initial threat recognition model identifies more threats, but has a high false positive rate.

[0070] The initial threat identification model is optimized according to the identification effect to obtain the target threat identification model.

[0071] AI-based cybersecurity threat detection system, including:

[0072] A collection module, used for collecting initial network flow data, and performing preprocessing operations on the initial network flow data to generate target network flow data;

[0073] A classification module, used to classify the target network traffic data and generate training network traffic data and test network traffic data;

[0074] A construction module, used to construct an initial threat identification model based on the training network traffic data, and continuously update the training network traffic data;

[0075] The optimization module is used to evaluate the recognition effect of the initial threat recognition model based on the test network traffic data, and optimize the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model.

[0076] Compared with related technologies, the network security threat detection method and system based on artificial intelligence provided by the present invention have the following beneficial effects:

[0077] The present invention generates target network traffic data by collecting initial network traffic data and performing preprocessing operations on the initial network traffic data; classifies the target network traffic data to generate training network traffic data and test network traffic data; constructs an initial threat identification model based on the training network traffic data; evaluates the recognition effect of the initial threat identification model based on the test network traffic data, and optimizes the initial threat identification model according to the recognition effect to obtain a target threat identification model, so that the network traffic data can be analyzed in real time through deep learning, machine learning and other technologies, and potential network security threats can be identified. The accuracy and efficiency of threat detection are improved by optimizing the processes of data preprocessing, feature selection and model training, while the false alarm rate is reduced, providing a more intelligent threat detection means for network security protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flowchart of the network security threat detection method based on artificial intelligence of the present invention;

[0079] Figure 2 A schematic diagram of data transmission by a network interface of the present invention;

[0080] Figure 3 is a schematic diagram of confusion matrix elements of the present invention;

[0081] Figure 4 This is a system block diagram of the artificial intelligence-based network security threat detection system of the present invention. DETAILED DESCRIPTION

[0082] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0083] Embodiment 1

[0084] like Figure 1 As shown, the network security threat detection method based on artificial intelligence includes:

[0085] S1, collecting initial network traffic data, and performing preprocessing operations on the initial network traffic data to generate target network traffic data;

[0086] S2, classifying the target network traffic data to generate training network traffic data and test network traffic data;

[0087] S3, building an initial threat identification model based on the training network traffic data, and continuously updating the training network traffic data;

[0088] S4, evaluating the recognition effect of the initial threat recognition model based on the test network traffic data, and optimizing the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model.

[0089] It is understandable that the network security threat detection method based on artificial intelligence of the present invention can improve the real-time response and processing efficiency of new threats in the network environment. Specifically, the entire detection process involves data collection and preprocessing, training based on algorithm models, and evaluation and optimization of model effects.

[0090] For data collection and preprocessing, we can first obtain the network data information flow of the target computer or network security device and process it accordingly. These data may contain data information from different sources in the network. The preliminary data cleaning step includes filtering invalid information and ensuring the consistency and integrity of the final data format to facilitate subsequent algorithm analysis and training.

[0091] By classifying the collected target network traffic data, we can obtain training network traffic data and test network traffic data. Based on the preprocessed training network traffic data, we can build an initial threat identification model, so that we can identify various malicious activity behavior patterns in different types of network environments and form a preliminary identification logic framework. In addition, in the early stage of model construction, we can choose normalized mathematical means to process the training network traffic data accordingly, so that all feature indicators are at a similar order of magnitude, which is convenient for subsequent model training and prediction, thereby ensuring the accuracy of the final model identification and improving its generalization performance.

[0092] Next, the initial threat identification model that has been built can be trained and optimized. Based on the test network traffic data, the effectiveness and stability of the model can be judged and evaluated, and corrected in a timely manner, so that the model can more efficiently and quickly identify and resist external attack threats in actual use. Through corresponding adjustments and optimizations, the overall performance of the overall identification system can be improved, and various known and unknown network attack threats can be identified more efficiently.

[0093] In addition, in order to improve the model's adaptive ability when facing unknown network security, the training network traffic data can be continuously updated and expanded. For example, when a new type of malicious program is discovered, various relevant data about such behavior can be quickly collected, and the existing training network traffic data can be updated. Then, the model can be improved and perfected accordingly in combination with artificial intelligence adaptive technology, so that it can more accurately and timely discover new security threats and take effective preventive measures in the shortest time.

[0094] In a specific implementation process, the initial network traffic data is collected, and a preprocessing operation is performed on the initial network traffic data to generate target network traffic data, specifically including:

[0095] Acquire a network interface, and collect initial network traffic data from the network interface;

[0096] Receiving the initial network traffic data, and performing an unpacking operation on the initial network traffic data to generate first network traffic data;

[0097] Based on a preset filtering rule, invalid information in the first network traffic data is removed to generate second network traffic data;

[0098] The data format of the second network traffic data is unified to generate the target network traffic data.

[0099] Among them, Figure 2 As shown in the figure, in the complex environment of network security analysis and monitoring, we can first identify the source of network traffic and collect data through network interfaces. These network interfaces can be Ethernet interfaces, wireless access points, etc., which are used for data transmission and can receive and send various data packets on the network in real time. Through network packet capture tools such as Wireshark, tcpdump, etc., we can capture the initial network traffic data flowing through these network interfaces and send them to the terminal device, and these data contain the original network communication records.

[0100] When preprocessing the captured initial network traffic data, the first step is to unpack the data. Specifically, unpacking means parsing the captured data packets from the original network byte stream and converting them into a format that is easier to understand and process, such as a hierarchical structure such as the IP layer, TCP / UDP layer, and application layer. Data unpacking can be achieved through network protocol analysis tools or programming libraries, such as Python's Scapy library, so that key information in the data packet can be accurately identified and extracted, such as the source address, destination address, port number, protocol type, etc., and then the first network traffic data can be generated.

[0101] In addition, in order to improve data quality, the first network traffic data can be filtered to remove invalid or redundant information. Based on conditions such as IP address, port range, protocol type and data packet size, the preset filtering rules refer to the data screening and filtering rules preset by the staff in advance. By accurately configuring the preset filtering rules, noise data can be effectively removed, such as non-critical network broadcasts, repeated requests or responses, etc., thereby generating the second network traffic data.

[0102] Finally, in order to ensure the consistency and comparability of the data, the second network traffic data can be processed in a unified format. This step involves data standardization, that is, converting data from different sources and formats into a unified format standard, such as JSON, XML, etc., and also involves operations such as renaming fields, converting data types, and processing missing values.

[0103] Through the above method, irrelevant noise can be removed, valid data can be retained, and target network traffic data can be generated, thereby ensuring that the data structure is clear and the format is unified, which is convenient for subsequent network security analysis, threat detection and traffic pattern recognition.

[0104] The constructing of the initial threat identification model based on the training network traffic data specifically includes:

[0105] Normalizing the training network traffic data to obtain input network traffic data;

[0106] The input network traffic data is trained to obtain the initial threat identification model.

[0107] The training network flow data is normalized to obtain input network flow data, which is processed by the following normalization formula:

[0108]

[0109] In the formula, Represents training network traffic data; Represents input network traffic data; Represents the minimum value of the training network traffic data; Indicates the maximum value of the training network traffic data; Indicates the minimum value of the target zoom range; Indicates the maximum value of the target zoom range.

[0110] It should be noted that in the initial stage of building an efficient and accurate initial threat identification model, the training network traffic data can be normalized so that the original training network traffic data can be adjusted to a uniform numerical range to eliminate the magnitude differences between data of different dimensions and improve the convergence speed and prediction accuracy of the model.

[0111] Specifically, normalization can map the training network traffic data to a new numerical space through a specific mathematical transformation to generate input network traffic data suitable for model training. Among them, the minimum and maximum values ​​in the training network traffic data can be used to determine the range of the original data, and the minimum and maximum values ​​of the target scaling range can be set according to the requirements of the model design and the expected output range. Through the normalization of the data, the original data can be linearly transformed into a new interval, so that the relative differences of the data can be retained and numerical problems caused by excessively large or small values ​​can be avoided.

[0112] The normalized input network traffic data can be used as training data for the initial threat identification model. During the model training phase, this data can be input into the selected neural network structure, and the model parameters can be continuously adjusted through iterative optimization algorithms until the model can accurately identify potential threat patterns in network traffic.

[0113] Through the above methods, the generalization ability of the model can be improved, and it can be ensured that the model can maintain stable performance when processing network traffic data of different scales and characteristics, which is convenient for real-time monitoring and defense against network attacks.

[0114] The training of the input network traffic data to obtain the initial threat identification model specifically includes:

[0115] Setting a neural network structure corresponding to the initial threat identification model, and determining an activation function type, a number of hidden layers, and a number of neurons of the neural network structure;

[0116] Set the maximum number of training times for the neural network structure and minimum total mean square error ;

[0117] Set the current iteration counter of the neural network structure to 0 and initialize the weight coefficients ,in, Indicates the number of layers; Indicates Layer neurons; Indicates Layer neurons;

[0118] For the input network traffic data, perform signal feedforward propagation and calculate the Layer The input signal of a neuron is:

[0119]

[0120] In the formula, Indicates Layer The input signal of each neuron; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer weight coefficient; Indicates Layer neurons; Indicates Layer neurons; Indicates Layer The actual output of each neuron;

[0121] when hour, and ,in, represents the bias term;

[0122] when hour, ,in, represents the activation function; when hour, .

[0123] It is understandable that when building and training the initial threat identification model, the corresponding neural network structure can be selected first, and the activation function type suitable for the characteristics of network traffic analysis can be selected. Then, based on the data complexity and computing resources, the number of hidden layers and the optimal configuration of neurons in each layer can be determined to balance the expressive power of the model with the risk of overfitting.

[0124] In order to monitor the training process and set the stopping condition, the maximum number of training times for the neural network structure can be determined, thereby preventing endless training from wasting time and resources. In addition, the minimum total mean square error can be defined as the threshold for training convergence, where the total mean square error is an indicator of the difference between the model's predicted value and the actual value. The smaller the value, the better the model fitting effect.

[0125] During the training initialization phase, the current iteration counter of the neural network structure can be reset to zero to record the training progress later, and then the weight coefficients can be assigned values ​​through the initialization method, and these weight coefficients connect the various neurons in the network.

[0126] Specifically, signal feedforward propagation can be performed for the input network traffic data, and the input signal and output value of the neuron can be calculated layer by layer, where the input signal of each neuron is the weighted sum of the output values ​​of all neurons in the previous layer. When calculating the input of the first layer of neurons, if there is external input, such as the original traffic data features, it can be directly used as the input signal; for the hidden layer and output layer neurons, their input signals can be determined based on the output of the previous layer, the weight coefficient and the activation function. The activation function can introduce nonlinear factors to the model, allowing the model to learn and recognize complex data patterns.

[0127] Through the above method, the weight coefficient of the model can be gradually adjusted to minimize the prediction error and achieve effective identification of potential threats in network traffic.

[0128] After the input network traffic data performs signal feedforward propagation, the error back propagation begins, specifically including:

[0129] Calculate the accumulation of the weight coefficients:

[0130]

[0131] In the formula, Indicates Accumulation of layer weight coefficients; represents the correction proportional coefficient of the weight, ; Indicates Layer The error term of each neuron; Indicates Layer The actual output of each neuron;

[0132] Jordi The layer is the output layer, and we get

[0133]

[0134] In the formula, Indicates Layer The error term of each neuron; Indicates Layer The input signal of each neuron; Indicates that the activation function is The derivative at ; Indicates Layer The expected output of each neuron; Indicates Layer The actual output of each neuron;

[0135] otherwise

[0136]

[0137] In the formula, Indicates Layer The error term of each neuron; Indicates Layer The input signal of each neuron; Indicates that the activation function is The derivative at ; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer The error term of each neuron; Indicates Layer weight coefficient;

[0138] Adjust the weight coefficient corresponding to the neural network structure:

[0139]

[0140] In the formula, Indicates the adjusted Layer weight coefficient; Indicates Layer weight coefficient; Indicates Accumulation of layer weight coefficients.

[0141] After adjusting the weight coefficient corresponding to the neural network structure, the number of iterations corresponding to the neural network structure is increased. , until the termination condition is met, the termination condition is:

[0142]

[0143] or

[0144]

[0145] In the formula, Indicates the current iteration number; Indicates the maximum number of training times; represents the total mean square error; Indicates the number of input signals; represents mathematical expectation; Indicates Layer The input signal of each neuron; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer The expected output of each neuron; Indicates Layer The output of a neuron; represents the minimum total mean square error;

[0146] The output of the neural network structure , which is the output of the initial threat identification model, where Indicates the probability of threat occurrence, and its value range is between 0 and 1.

[0147] In practical applications, for input network traffic data, the feedforward propagation process of the signal can be performed first, which involves the sequential transmission of data through each layer of the neural network until it reaches the output layer. In order to optimize network performance, that is, to reduce prediction errors, the error backpropagation algorithm can be started, which updates the weight coefficients in the network by calculating and propagating error terms. And this process is iterative, and the number of iterations can be increased and counted after each iteration until the preset termination condition is met. The termination condition includes that the number of iterations reaches the maximum number of training times or the total mean square error is lower than the preset minimum value. By training the neural network structure, the prediction result of the threat identification model can be output, that is, the probability of threat occurrence, whose value is between 0 and 1, reflecting the possibility of potential threats in the input network traffic data.

[0148] The step of evaluating the recognition effect of the initial threat recognition model based on the test network traffic data, and optimizing the initial threat recognition model according to the recognition effect to obtain a target threat recognition model specifically includes:

[0149] Obtaining a confusion matrix corresponding to the test network traffic data, and evaluating the recognition effect of the initial threat recognition model according to the confusion matrix;

[0150] like Figure 3 As shown, the elements of the confusion matrix include true positive examples, false positive examples, true negative examples and false negative examples, wherein the true positive examples are test network traffic data correctly identified as threats; the false positive examples are test network traffic data incorrectly identified as threats; the true negative examples are test network traffic data correctly identified as normal; and the false negative examples are test network traffic data incorrectly identified as normal.

[0151] Based on the confusion matrix, calculating the performance indicators of the initial threat identification model, wherein the performance indicators include accuracy, precision, recall, F1 score and false positive rate;

[0152] The calculation formula of the accuracy is:

[0153]

[0154] In the formula, Represents the accuracy of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents a true counterexample, i.e., the test network traffic data that is correctly identified as normal; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; represents a false negative example, i.e., test network traffic data that is mistakenly identified as normal;

[0155] The calculation formula of the accuracy is:

[0156]

[0157] In the formula, Represents the accuracy of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat;

[0158] The calculation formula of the recall rate is:

[0159]

[0160] In the formula, represents the recall rate of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents a false negative example, i.e., test network traffic data that is mistakenly identified as normal;

[0161] The calculation formula of the F1 score is:

[0162]

[0163] In the formula, represents the F1 score of the initial threat identification model; Represents the accuracy of the initial threat identification model; represents the recall rate of the initial threat identification model;

[0164] The calculation formula of the false positive rate is:

[0165]

[0166] In the formula, represents the false positive rate of the initial threat identification model; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; represents a true counterexample, i.e., the test network traffic data that is correctly identified as normal;

[0167] The performance indicators are analyzed, and the precision and recall are weighed to evaluate the recognition effect of the initial threat recognition model. That is, a high precision and a low recall rate indicate that the initial threat recognition model rarely misjudges normal traffic as a threat, but sometimes misses a real threat; a low precision and a high recall rate indicate that the initial threat recognition model identifies more threats, but has a high false positive rate.

[0168] The initial threat identification model is optimized according to the identification effect to obtain the target threat identification model.

[0169] It can be understood that first, the test network traffic data can be collected and processed to obtain its corresponding confusion matrix, and the confusion matrix includes four elements: true positive examples (TP), false positive examples (FP), true negative examples (TN) and false negative examples (FN), which represent four possible results of data threat identification, namely, TP represents data correctly identified as threats, FP represents data incorrectly identified as threats, TN represents data correctly identified as normal, and FN represents data incorrectly identified as normal.

[0170] According to the confusion matrix and its elements, the corresponding key performance indicators can be calculated to comprehensively evaluate the model performance. Accuracy, as the most intuitive indicator, can reflect the correctness of the overall classification of the model, which measures the model's ability to classify the overall data. Precision can reflect the accuracy of threat identification, and a high precision rate means that the model has fewer misjudgments when identifying threats. Recall can reflect the model's ability to identify all real threats, and a high recall rate means that the model can discover more potential threats. The F1 score, as the harmonic mean of precision and recall, can balance precision and recall, and is an important indicator for evaluating the comprehensive performance of the model. In addition, the false positive rate (FPR) can reflect the rate at which the model misclassifies normal data as threats. A low false positive rate of the model can reduce unnecessary alarms and reduce system burden.

[0171] After analyzing the performance indicators, you can weigh the relationship between precision and recall based on the corresponding business needs and security policies. For example, when pursuing an extremely low false alarm rate (low FPR), you may sacrifice some recall; while in the case of zero tolerance for security threats, you may prefer to increase the recall, but this may result in a higher false alarm rate.

[0172] Through the above method, based on the evaluation results, through the optimization algorithms in machine learning, such as gradient descent, parameter adjustment, feature selection and model fusion, the initial threat identification model can be optimized to improve its overall performance, achieve the optimal balance point on key performance indicators, and construct a target threat identification model, thereby improving the accuracy and efficiency of the model in identifying network threats, facilitating subsequent network security defense.

[0173] Embodiment 2

[0174] like Figure 4 As shown, the network security threat detection system based on artificial intelligence includes:

[0175] A collection module, used for collecting initial network flow data, and performing preprocessing operations on the initial network flow data to generate target network flow data;

[0176] A classification module, used to classify the target network traffic data and generate training network traffic data and test network traffic data;

[0177] A construction module, used to construct an initial threat identification model based on the training network traffic data, and continuously update the training network traffic data;

[0178] The optimization module is used to evaluate the recognition effect of the initial threat recognition model based on the test network traffic data, and optimize the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model.

[0179] Through the introduction of the above embodiments, the present invention can collect initial network traffic data through an artificial intelligence-based network security threat detection method and system, and perform preprocessing operations on the initial network traffic data to generate target network traffic data; classify the target network traffic data to generate training network traffic data and test network traffic data; construct an initial threat identification model based on the training network traffic data; evaluate the recognition effect of the initial threat identification model based on the test network traffic data, and optimize the initial threat identification model according to the recognition effect to obtain a target threat identification model, so that the network traffic data can be analyzed in real time through deep learning, machine learning and other technologies, and then potential network security threats can be identified, and the accuracy and efficiency of threat detection are improved by optimizing the process of data preprocessing, feature selection and model training, while reducing the false alarm rate, providing a more intelligent threat detection means for network security protection.

[0180] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0182] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A network security threat detection method based on artificial intelligence, characterized in that: include: Collecting initial network traffic data, and performing preprocessing operations on the initial network traffic data to generate target network traffic data; Classifying the target network traffic data to generate training network traffic data and test network traffic data; Building an initial threat identification model based on the training network traffic data, and continuously updating the training network traffic data; Evaluate the recognition effect of the initial threat recognition model based on the test network traffic data, and optimize the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model; The constructing of the initial threat identification model based on the training network traffic data specifically includes: Normalizing the training network traffic data to obtain input network traffic data; Training the input network traffic data to obtain the initial threat identification model; The training of the input network traffic data to obtain the initial threat identification model specifically includes: Setting a neural network structure corresponding to the initial threat identification model, and determining an activation function type, a number of hidden layers, and a number of neurons of the neural network structure; Set the maximum number of training times for the neural network structure and minimum total mean square error ; Set the current iteration counter of the neural network structure to 0 and initialize the weight coefficients ,in, Indicates the number of layers; Indicates Layer neurons; Indicates Layer neurons; For the input network traffic data, perform signal feedforward propagation and calculate the Layer The input signal of a neuron is: In the formula, Indicates Layer The input signal of each neuron; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer weight coefficient; Indicates Layer neurons; Indicates Layer neurons; Indicates Layer The actual output of each neuron; when hour, and ,in, represents the bias term; when hour, ,in, represents the activation function; when hour, ; After the input network traffic data performs signal feedforward propagation, the error back propagation begins, specifically including: Calculate the accumulation of the weight coefficients: In the formula, Indicates Accumulation of layer weight coefficients; represents the correction proportional coefficient of the weight, ; Indicates Layer The error term of each neuron; Indicates Layer The actual output of each neuron; Jordi The layer is the output layer, and we get In the formula, Indicates Layer The error term of each neuron; Indicates Layer The input signal of each neuron; Indicates that the activation function is The derivative at ; Indicates Layer The expected output of each neuron; Indicates Layer The actual output of each neuron; otherwise In the formula, Indicates Layer The error term of each neuron; Indicates Layer The input signal of each neuron; Indicates that the activation function is The derivative at ; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer The error term of each neuron; Indicates Layer weight coefficient; Adjust the weight coefficient corresponding to the neural network structure: In the formula, Indicates the adjusted Layer weight coefficient; Indicates Layer weight coefficient; Indicates Accumulation of layer weight coefficients; After adjusting the weight coefficient corresponding to the neural network structure, the number of iterations corresponding to the neural network structure is increased. , until the termination condition is met, the termination condition is: or In the formula, Indicates the current iteration number; Indicates the maximum number of training times; represents the total mean square error; Indicates the number of input signals; represents mathematical expectation; Indicates Layer The input signal of each neuron; Indicates The number of neurons in the layer; Σ represents the summation symbol; Indicates Layer The expected output of each neuron; Indicates Layer The output of a neuron; represents the minimum total mean square error; The output of the neural network structure , which is the output of the initial threat identification model, where Indicates the probability of threat occurrence, and its value range is between 0 and 1; The evaluating the recognition effect of the initial threat recognition model based on the test network traffic data, and optimizing the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model, specifically includes: Obtaining a confusion matrix corresponding to the test network traffic data, and evaluating the recognition effect of the initial threat recognition model according to the confusion matrix; The elements of the confusion matrix include true positive examples, false positive examples, true negative examples and false negative examples, wherein the true positive examples are test network traffic data correctly identified as threats; the false positive examples are test network traffic data incorrectly identified as threats; the true negative examples are test network traffic data correctly identified as normal; and the false negative examples are test network traffic data incorrectly identified as normal. Based on the confusion matrix, calculating the performance indicators of the initial threat identification model, wherein the performance indicators include accuracy, precision, recall, F1 score and false positive rate; The calculation formula of the accuracy is: In the formula, Represents the accuracy of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents a true counterexample, i.e., the test network traffic data that is correctly identified as normal; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; represents a false negative example, i.e., test network traffic data that is mistakenly identified as normal; The calculation formula of the accuracy is: In the formula, Represents the accuracy of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; The calculation formula of the recall rate is: In the formula, represents the recall rate of the initial threat identification model; represents true positives, i.e., test network traffic data that is correctly identified as a threat; represents a false negative example, i.e., test network traffic data that is mistakenly identified as normal; The calculation formula of the F1 score is: In the formula, represents the F1 score of the initial threat identification model; Represents the accuracy of the initial threat identification model; represents the recall rate of the initial threat identification model; The calculation formula of the false positive rate is: In the formula, represents the false positive rate of the initial threat identification model; represents false positives, i.e., test network traffic data that is incorrectly identified as a threat; represents a true counterexample, i.e., the test network traffic data that is correctly identified as normal; The performance indicators are analyzed, and the precision and recall are weighed to evaluate the recognition effect of the initial threat recognition model. That is, a high precision and a low recall rate indicate that the initial threat recognition model rarely misjudges normal traffic as a threat, but sometimes misses a real threat; a low precision and a high recall rate indicate that the initial threat recognition model identifies more threats, but has a high false positive rate. The initial threat identification model is optimized according to the identification effect to obtain the target threat identification model.

2. The network security threat detection method based on artificial intelligence according to claim 1 is characterized in that: The collecting of initial network traffic data and performing preprocessing operations on the initial network traffic data to generate target network traffic data specifically includes: Acquire a network interface, and collect initial network traffic data from the network interface; Receiving the initial network traffic data, and performing an unpacking operation on the initial network traffic data to generate first network traffic data; Based on a preset filtering rule, invalid information in the first network traffic data is removed to generate second network traffic data; The data format of the second network traffic data is unified to generate the target network traffic data.

3. The network security threat detection method based on artificial intelligence according to claim 1 is characterized in that: The training network flow data is normalized to obtain input network flow data, which is processed by the following normalization formula: In the formula, Represents training network traffic data; Represents input network traffic data; Represents the minimum value of the training network traffic data; Indicates the maximum value of the training network traffic data; Indicates the minimum value of the target zoom range; Indicates the maximum value of the target zoom range.

4. An artificial intelligence-based network security threat detection system, applied to the artificial intelligence-based network security threat detection method according to any one of claims 1 to 3, the detection system comprising: A collection module, used for collecting initial network flow data, and performing preprocessing operations on the initial network flow data to generate target network flow data; A classification module, used to classify the target network traffic data and generate training network traffic data and test network traffic data; A construction module, configured to construct an initial threat identification model based on the training network traffic data, and continuously update the training network traffic data; The optimization module is used to evaluate the recognition effect of the initial threat recognition model based on the test network traffic data, and optimize the initial threat recognition model according to the recognition effect based on artificial intelligence adaptive technology to obtain a target threat recognition model.

Images