A method for detecting internal network security threats

Abstract

This invention relates to the field of network security anomaly detection technology, specifically to a method for detecting internal network security threats. Addressing the problems of uneven data distribution in internal threat detection datasets, the inability of existing anomaly-based threat detection methods to correlate with specific threat scenarios, and low detection efficiency due to insufficient labeled data, this invention uses a DNN-based Prototype network to perform small-sample feature learning on data from multiple attack scenarios. It also uses a CWGAN network to augment the sample data, and finally employs distance similarity to classify various data types. Furthermore, a GA genetic algorithm is used to automatically optimize the parameters of the entire algorithm. This invention improves the detection performance of the network security internal threat detection method.

Description

Technical Field

[0001] This invention relates to the field of network security anomaly detection technology, and in particular to a method for detecting internal network security threats. Background Technology

[0002] In recent years, security incidents caused by malicious operations by internal users have occurred frequently. Moreover, since internal users mostly have access to the system, know the vulnerabilities of the internal network, and have access to core information, internal attacks often cause more serious losses than external attacks.

[0003] While some research has been achieved in the field of internal user behavior detection, problems such as uneven distribution of datasets, lack of sufficient labeled data, and difficulty in detecting unknown threat behaviors still exist, resulting in poor detection performance. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting internal network security threats. It uses few-shot learning for modeling, and a DNN-based Prototype network to learn features from limited data and perform classification and detection. At the same time, it uses a CWGAN network for sample augmentation to improve the model's detection performance.

[0005] To achieve the above objectives, the present invention provides a method for detecting internal network threats, comprising the following steps:

[0006] Perform feature collection on the raw data to obtain a dataset;

[0007] The dataset is divided into a support set and a query set;

[0008] Data is input into the Prototype network for internal testing;

[0009] Based on CWGAN network data augmentation, the generated data is expanded into the query set to complete the detection task.

[0010] In the process of collecting features from the raw data and obtaining the dataset, behavioral features are first extracted from the raw data, and then the behavioral feature data undergoes preprocessing for missing values ​​and normalization.

[0011] The support set is the training set, the query set is the test set, and both the support set and the query set are K-wayN-shot datasets.

[0012] The process of inputting data into the Prototype network for internal detection involves inputting the data into the mapping function in the Prototype network to obtain the feature vector of each sample, then calculating the prototype representation of the corresponding class from the feature vector, and subsequently using the feature vector in the query set and the calculated prototype for classification verification.

[0013] In the process of augmenting the generated data to the query set based on the CWGAN network to complete the detection task, the category label of the prototype representation calculated during internal detection is used as the input constraint condition, so that the generated data of the trained GAN network is infinitely similar to the source input data, and the generated data is expanded to the query set to complete the classification verification test.

[0014] The parameter adjustment of the Prototype network and CWGAN network data is achieved through the GA genetic algorithm.

[0015] Specifically, during the internal detection process of inputting data into the Prototype network, a DNN neural network is used to learn the feature representation of the sample data.

[0016] This invention provides a method for detecting internal network security threats. Addressing the problem of uneven data distribution, the inability to effectively use deep networks to learn data features, and the inability to correlate specific threat behaviors or scenarios, this invention employs few-shot learning for modeling. It also uses a DNN-based Prototype network to learn features and perform classification detection on limited data. Furthermore, to address the issue of insufficient labeled data, a CWGAN network is used, along with a GA genetic algorithm for automatic optimization, thereby improving the model's detection performance. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a network security internal threat detection method according to the present invention.

[0019] Figure 2 This is a schematic diagram of the network composition framework of a network security internal threat detection method according to the present invention.

[0020] Figure 3 This is a schematic diagram illustrating the principle of internal threat detection using the Prototype network of this invention.

[0021] Figure 4 This is a schematic diagram of the objective function and network structure of the CWGAN network of the present invention.

[0022] Figure 5 This is a comparison chart of the evaluation index parameters of this invention and other algorithms. Detailed Implementation

[0023] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0024] Please see Figure 1 This invention proposes a method for detecting internal network security threats, comprising the following steps:

[0025] S1: Collect features from the raw data to obtain the dataset;

[0026] S2: Divide the dataset into a support set and a query set;

[0027] S3: Data is input into the Prototype network for internal testing;

[0028] S4: Based on CWGAN network data augmentation, the generated data is expanded into the query set to complete the detection task.

[0029] In the process of collecting features from raw data and obtaining the dataset, behavioral features are first extracted from the raw data, and then the behavioral feature data undergoes preprocessing for missing values ​​and normalization.

[0030] The support set is the training set, and the query set is the test set. Both the support set and the query set are K-wayN-shot datasets.

[0031] The process of inputting data into the Prototype network for internal detection involves inputting the data into the mapping function in the Prototype network to obtain the feature vector of each sample, and then calculating the prototype representation of the corresponding class from the feature vector. Subsequently, the feature vector in the query set and the calculated prototype will be used for classification verification.

[0032] In the process of augmenting the generated data to the query set based on the CWGAN network to complete the detection task, the category label of the prototype representation calculated during internal detection is used as the input constraint condition, so that the generated data of the trained GAN network is infinitely similar to the source input data. The generated data is then expanded to the query set and the classification verification test is completed.

[0033] The parameter tuning of Prototype and CWGAN network data is achieved through the GA genetic algorithm.

[0034] During the internal detection process of inputting data into the Prototype network, a DNN neural network is used to learn the feature representation of the sample data.

[0035] See Figure 2 This allows us to understand the network composition framework of the aforementioned network security internal threat detection method.

[0036] The present invention will be further described below with reference to specific embodiments and execution steps:

[0037] This embodiment uses version 4.2 of the CMU-CERT dataset for algorithm training and testing. The CERT dataset defines malicious activity by generating various use cases. This dataset contains 32.77 million unique events, and available audit data sources include login activity, email traffic, web browsing tracking, file access logs, USB drive usage, and LDAP information describing organizational hierarchies and user roles. Version 4.2 of the dataset records activity logs from 1,000 employees from different organizations over 17 months.

[0038] 1. Data Processing

[0039] Based on the log information in the dataset, the CERT dataset defines three threat scenarios, as follows:

[0040] (1) Scenario 1: A user who had never used a removable drive or worked overtime before logged in outside of office hours using a removable drive and uploaded data to wikileaks.org. Shortly thereafter, the user left the organization.

[0041] (2) Scenario 2: The user begins searching for new job opportunities through job search websites. Before leaving the organization, the user leaks confidential information using a USB drive. The USB drive is used more frequently than he / she previously used.

[0042] (3) Scenario 3: A user downloads a key logger and obtains a list of passwords for different employees within the organization. Next, he / she uses a USB drive to transfer this password list to the supervisor's computer and attempts to search for the supervisor's password. Once successful, he / she logs into the supervisor's computer and sends an email that causes internal panic. This type of malicious activity often originates from system administrators, especially when they have conflicts with their supervisors.

[0043] This invention will conduct internal threat detection and analysis based on these three types of threat scenarios. It will mainly extract relevant features from the five log files of the dataset: login, device, file, email, and http. The extracted features are shown in Table 1 below.

[0044] Table 1 Behavioral Characteristics Table

[0045]

[0046]

[0047] A total of 20 behavioral features were extracted from the 5 log files in the table. Then, these feature data were preprocessed with missing values ​​and normalization to reduce the complexity of network training.

[0048] 2. Internal threat detection based on Prototype networks

[0049] After completing the feature collection of the raw data, the data is divided into a support set and a query set:

[0050] (1) Support set: i.e., training set. Suppose there is a sample set S = {(x1, y1), ..., (x... y1)} containing N labels. N ,y N )}, where x represents sample data, y i ∈{1,...,K} represents the label of the class corresponding to the sample data, S K This represents a dataset with k classes of samples. This dataset is called the K-way N-shot dataset, which contains a total of K*N samples from K classes.

[0051] (2) Query set: also known as test set, which corresponds to the data volume of K-wayN-shot as the support set, but unlike the support set, some data in the query set is unlabeled.

[0052] After dividing the dataset into a support set and a query set, the prototype network will train an embedding function f. θ (x), where x represents sample data, is a neural network that represents the projection of sample x into the metric space. The parameter θ is a value learned by the neural network. By learning a better θ value, the distribution of samples in the metric space can be described more accurately, that is, making data of the same category closer and data of different categories farther apart. This invention will use a DNN neural network to learn the feature representation of sample data. Then, the prototype representation of each class is calculated on this metric space. The prototype calculation formula is as follows:

[0053]

[0054] Where c k S represents the prototype representation of category k. k Represents category k, |S k The number of samples in category k, (x i ,y i ) represents the feature vector and label of the sample.

[0055] The softmax distance function calculates the distance distribution between samples in the query set and their prototypes, thereby determining which prototype a sample in the query set belongs to and completing the classification task. The formula for this distribution function is as follows:

[0056]

[0057] This formula expresses the probability that a sample x belongs to category k through relevant calculations, where d(·) is the distance function. This paper chooses the cosine similarity function as the distance function.

[0058] Finally, the prototype network expresses the negative logarithmic probability of the distribution function as its loss function, as shown in the following formula. This formula is used to find its minimum value through gradient descent, thus learning a good θ value after convergence. At this point, it can be considered that the function f has been trained successfully. θ (x) can map the same samples to places that are close to each other.

[0059] J(θ)=-log(p θ (y=k|x))

[0060] The Prototype network's method for internal threat detection can be summarized simply as follows: Figure 3 As shown, the samples are projected into a metric space, and the prototype to which each sample belongs is calculated, such as C1, C2, and C3 in the figure. Then, the detected sample X is projected into this metric space, and finally, the class to which the sample belongs is calculated using a distance function.

[0061] 3. Data Augmentation Based on CWGAN

[0062] This module focuses on sample augmentation under small sample conditions. Given a small sample training task τ = (S... τ Q τ ) by support set S τ and query set Q τ Composition, where (x,y)∈S τ Let x and y be the sample data and label data, respectively. Then, sample x is processed through a feature extraction network, i.e., the embedding function f. θ (x) generates a representation vector The representation vector is used to compute its prototype representation c through a prototype network. The label y of prototype c is then used as a conditional input to the generator of the WGAN network to obtain synthetic sample data. Where z represents the noise data, the generator aims to synthesize data that is as close as possible to the real data, and the discriminator strives to distinguish the synthesized data. Finally, the data synthesized by the trained generator is added to the training dataset to complete the sample augmentation task. The objective function and network structure of CWGAN are as follows: Figure 4 As shown.

[0063]

[0064] The entire network model consists of a prototype network and a CWGAN network. During model training, the GA genetic algorithm is used to adjust the optimal parameters to continuously reduce the network's objective function. Finally, the detection task is completed and various indicators are calculated using the test dataset, which is also the query set.

[0065] Furthermore, this invention compares the internal threat detection methods based on LSTM-AE, DCNNs, and GCN with the detection method proposed in this chapter. Precision, Recall, and F1-score metrics were calculated from a macro-average perspective to compare and validate the proposed model. Figure 5 The results calculated for each evaluation indicator are shown.

[0066] Significant differences can be observed among the several algorithms across the four evaluation metrics. In terms of accuracy, DCNNs perform best, followed by the proposed method. However, DCNNs' performance is less than satisfactory in other metrics. In terms of precision, LSTM-AE is the best, followed by the proposed method. In the remaining two metrics, recall and F1 score, the proposed method outperforms the other algorithms. Therefore, the proposed network security insider threat detection method is effective in detecting insider threats.

[0067] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A method for detecting insider threats in network security, characterized by, The method comprises the following steps: characteristics of original data are collected to obtain a data set; the data set is divided into a support set and a query set; the support set is a training set, and the query set is a test set, both of which are K-way N-shot data sets; data is input into a Prototype network for internal detection; the process of inputting data into the Prototype network for internal detection is as follows: the data is input into a mapping function in the Prototype network to obtain a feature vector of each sample, and a prototype representation of a corresponding class is calculated based on the feature vector; subsequently, the feature vector in the query set is used for classification verification with the calculated prototype; in the process of inputting data into the Prototype network for internal detection, a DNN neural network is used to learn a feature representation of sample data; a distance distribution between samples in the query set and prototypes is calculated based on a softmax distance function, so as to determine which class the sample in the query set belongs to, and complete a classification task; a distribution function calculation formula is as follows: a probability of a sample x belonging to a class k is obtained through relevant calculation, wherein d(·) is a distance function; based on CWGAN network data enhancement, generated data is expanded to the query set to complete a detection task.

2. The network security internal threat detection method according to claim 1, wherein in the process of collecting characteristics of original data to obtain a data set, behavior characteristics are first extracted from the original data, and then the behavior characteristic data is preprocessed by filling in missing values and normalization.

3. The network security internal threat detection method according to claim 1, wherein in the process of completing a detection task based on CWGAN network data enhancement and expanding generated data to the query set, a class label of the calculated prototype representation is input as a constraint condition to make the generated data of the trained GAN network infinitely similar to source input data, and the generated data is expanded to the query set to complete classification verification testing.

4. The method of detecting insider threats to network security of claim 1, wherein, Parameter adjustment of the Prototype network and CWGAN network data is realized through a GA genetic algorithm.

Images