An unknown threat self-adaptive detection method based on deceptive defense

Abstract

The application discloses an unknown threat adaptive detection method based on deceptive defense, and adopts an unknown threat adaptive flow detection system based on a deceptive defense technology to realize the method, wherein the system comprises a malicious flow detection module, a honeypot module and an adaptive detection upgrade module; the malicious flow detection module discovers attacks in a network; the honeypot module acquires unknown threat flows in the network; the adaptive detection upgrade module analyzes and learns the unknown threat flows in the honeypot module, and updates a learned model into the malicious flow detection module. The application integrates the deceptive defense technology and the malicious flow detection technology, captures unknown threats through the deceptive defense technology, uses an improved data set balancing algorithm to expand the unknown threats belonging to a small sample, enables a malicious flow recognition algorithm to recognize the unknown threats, adaptively expands attack modes that can be recognized, and improves the universality of the malicious flow detection system.

Description

Technical Field

[0001] This invention relates to the field of industrial control security, and specifically to an adaptive detection method for unknown threats based on deception defense. Background Technology

[0002] As a product of the deep integration of next-generation network information technology and manufacturing, the Industrial Internet is a crucial infrastructure and key technological support for realizing the digital, networked, and intelligent development of industries, and is widely regarded as an important cornerstone of the Fourth Industrial Revolution. In recent years, the continuous improvement of 5G infrastructure construction and the ongoing research, development, and promotion of new technologies and applications integrated with Industrial Internet technologies have brought tremendous opportunities to the development of the Industrial Internet, while also posing serious challenges.

[0003] Many attacks exploit the Industrial Internet (IIoT) to compromise industrial control systems, necessitating security systems to detect and detect malicious traffic within the IIoT. However, with the rapid development of technology, attack methods are constantly evolving, and detection systems cannot keep pace with these advancements. Deceptive defense technologies can redirect and capture attack traffic, preventing legitimate traffic from triggering attacks while simultaneously targeting them. Threats captured by deceptive defense technologies but unrecognized by existing detection systems are termed unknown threats. These unknown threats often represent significant potential threats to the network; improving the detection capabilities for these attacks can greatly enhance overall network security.

[0004] Deceptive defense technology is a highly deceptive security protection technique that can operate on various networks and endpoint systems with intentionally left-signature vulnerabilities. It can induce intruders to launch attacks, thereby capturing the attack source and achieving security defense, protecting critical system endpoints from harm. The attack methods used by unknown threats are usually undisclosed, and these threats are typically undetectable by detection systems. Deceptive defense technology can disguise itself as important assets, luring these threats to attack, and then capturing these attacks. These unknown threats are characterized by small sample sizes, high stealth, and high severity.

[0005] Automated analysis and defense against unknown threats compensate for the lag in manual analysis and enhance the defense capabilities of existing technologies against unknown attacks. Attack traffic in the network is captured through deceptive defense techniques. This captured traffic is analyzed and classified, and unanalyzable attack traffic is added to the malicious traffic detection system's dataset. When unknown threats are added to the dataset, an improved dataset balancing algorithm is used to prevent imbalanced datasets from negatively impacting the malicious traffic detection system. By updating the dataset, the malicious traffic detection system updates its original detection model, enabling it to detect an increase in attack types and adaptively discover potential attack traffic in the network, reducing human intervention.

[0006] This invention relates to the discovery and detection of unknown threats in a local area network (LAN) within an industrial control system. It utilizes deceptive defense technology to guide and capture attack traffic, detects and filters the captured attack traffic, adaptively detects unknown threats, and automatically upgrades the traffic detection system in the LAN, thereby achieving the identification of unknown threats.

[0007] (1) An adaptive traffic detection system for unknown threats based on deception defense technology (hereinafter referred to as the adaptive detection system) is proposed. This adaptive detection system consists of a malicious traffic detection module, a honeypot module, and an adaptive detection upgrade module. The malicious traffic detection module (one module in the adaptive detection system) promptly detects attacks in the network. The honeypot module acquires unknown threat traffic in the network. The adaptive detection upgrade module analyzes and learns from the unknown threat traffic in the honeypot module, updating the learned model in the malicious traffic detection module. Thus, the adaptive detection system can analyze and detect unknown threats in the local area network.

[0008] (2) An improved dataset balancing algorithm was implemented. Minority class samples were augmented using the average distance between nearest minority classes and the number of nearest majority classes, ensuring a balanced sample size across all classes in the dataset and preventing the impact of an imbalanced dataset on the deep learning algorithm. This algorithm is used in the adaptive detection upgrade module mentioned above as the core algorithm for analyzing and learning unknown threat traffic. Summary of the Invention

[0009] To address the shortcomings in current malicious traffic detection methods for industrial control systems, this invention proposes an adaptive unknown threat detection method based on deception defense. This method employs an adaptive unknown threat traffic detection system based on deception defense technology. A schematic diagram of the system's deployment in an existing local area network is shown below. Figure 1As shown in the diagram. The industrial control equipment, routers, and switches are existing devices, while the gateway server and honeypots are devices equipped with the system of this invention. The malicious traffic detection module and adaptive detection upgrade module of the adaptive traffic detection system are deployed in the gateway server. Each honeypot device is equipped with the honeypot module of this system, capable of feeding back captured attack traffic to the gateway server. Simultaneously, each honeypot device possesses existing deception defense technologies, capable of guiding and capturing unknown threats (this invention operates within existing local area network devices, where the gateway server and honeypots are the carriers of this invention; this invention is software).

[0010] One or more honeypot devices should be deployed in industrial control networks. The malicious traffic detection module in this system performs threat detection using an existing neural network model. The adaptive detection upgrade module upgrades and updates the malicious traffic detection module by retraining the neural network model, improving the detection accuracy of the adaptive traffic detection system and increasing the types of threats the system can identify.

[0011] Compared to normal traffic, attack traffic accounts for a smaller proportion. When unknown threats are detected, the principles and methods of these attacks are usually undisclosed, possessing strong stealth and targeting capabilities, and are generally less frequent. Therefore, when unknown threat data is integrated into the original dataset, the dataset sample space will be relatively isolated and discrete. This can lead to overfitting during the training of the neural network model (the neural network model is for artificial intelligence detection, the subject of this invention). Therefore, when unknown threats are integrated into the dataset, sample balancing is necessary. The dataset initially carried by the malicious traffic detection module is called the original dataset, denoted as D. org .

[0012] The adaptive detection method for unknown threats based on deception defense proposed in this invention specifically includes the following steps:

[0013] S1. Construct an unknown threat adaptive traffic detection system based on deception defense technology as described above.

[0014] S2. Analyze the threat traffic captured by the honeypot module using the malicious traffic detection module to obtain unknown threat traffic. Create a preprocessed dataset D from the unknown threat traffic. pre .

[0015] S3, preprocess the dataset D pre and the original dataset D org The merged dataset D is obtained. cbn * For the merged dataset D cbn * Clustering is performed on the sample space to obtain the clustered dataset D. cbn and determine dataset Dcbn The division of the dataset into minority and majority classes. Below is an example of this division in the merged dataset D. cbn * The formula for clustering and the clustered dataset D cbn process.

[0016] Before performing clustering operations, D cbn * D consists of two parts org and D pre D cbn Central Plains belongs to D org The sample points have a complete set of cluster identifiers C. old , in, Let represent the position of the center point of the k-th class, k = 1, 2, ..., S, and the total number of classes in the original dataset is S. D cbn Central Plains belongs to D pre Each sample point is treated as an independent cluster. The D function is completed using the following formula. cbn * Clustering and obtaining D cbn The new cluster identifier set C new Formula to calculate D cbn * If the distance between all clusters is less than a pre-set threshold η, then the two clusters will be merged.

[0017] The formula for calculating cluster distance is as follows:

[0018]

[0019] Among them, C a and C b Let x and y represent two arbitrary clusters, C and C, respectively. a and C b The sample points in |C a |and|C b | represents the total number of samples in the cluster. If d(C a C b If the sample size is less than η, the two clusters are merged, and this process is repeated until no further merging is possible. The resulting clusters are then sorted from largest to smallest according to the number of samples. There are N new clusters. This gives us the clustered dataset D. cbn .

[0020] The calculation process shows that new sample points are aggregated into two places: first, into the existing S categories, and second, into new categories. Due to the small number of unknown threat samples, the first S known categories are the majority, while the latter NS categories are the minority. Unknown threats are concentrated in the minority categories.

[0021] S4. For the clustered dataset D... cbn The average distance d between the nearest minority classes in the sample space sa The number N of nearest neighbor majority classes ma ld is a measure of the learning complexity of the minority class samples.

[0022] For a certain minority class C i The calculation process for the average distance between neighboring minority classes is as follows. Where, C i For set A certain class in the set, where the value of i ranges from S+1 to N integers. (C) i Average distance to m neighboring minority classes C i The average distance between the nearest minority class and the nearest neighbor class represents the sparsity of that class in the minority class space. Here, m is a pre-set constant. The m nearest minority classes represent C. i The top m minority classes are sorted by their distance from all other minority classes in ascending order.

[0023] For a certain minority class C i The calculation process for the number of local nearest neighbor majority classes is as follows. C i The circle formed by the distance to the m-th nearest neighbor (minority class) has the following radius: The number of majority class samples within this circle is... Indicate C i The number of local nearest neighbor majority classes.

[0024] Average distance of neighboring minority classes A larger value indicates that the minority class samples are sparsely distributed, making them difficult to learn from, and thus requiring higher learning weights. Conversely, a denser distribution makes learning easier and reduces learning complexity.

[0025] Number of local nearest neighbor majority classes A larger value indicates that the sample class is located on the boundary or in the center of the majority sample class, meaning the learning cost for that class will be high because it will be influenced by the majority sample class. Conversely, a smaller value indicates lower learning complexity and easier learning.

[0026] The formula for normalizing the mean distance between nearest neighbors and minority classes is as follows:

[0027]

[0028] Where j takes the value of a positive integer from S+1 to N. This represents the average distance between the j-th minority class and its nearest neighboring minority classes. This represents the normalized value of the average distance between the nearest minority classes of the i-th minority class.

[0029] The formula for normalizing the number of local nearest neighbor majority classes is as follows:

[0030]

[0031] This represents the number of local nearest neighbor majority classes for the j-th local neighborhood. This represents the normalized value of the number of local nearest neighbor majority classes for the i-th local nearest neighbor.

[0032] Then for the minority class C i learning complexity ld i The calculation formula is as follows:

[0033]

[0034] Where α represents the weighting coefficient, which balances the relationship between the average distance of the nearest minority class and the number of local nearest majority classes. The learning complexity is ld. i The larger the value, the higher the value of the sample class C. i The more difficult it is to learn, the more neighboring samples need to be generated.

[0035] S5. For the clustered dataset D... cbn Based on the learning complexity ld, calculate the number of minority class samples synthesized. Generate new minority class samples using the synthesis formula, and merge the generated minority class samples into D. cbn This forms a new sample dataset D. nrb Dataset D nrb The sample space tends to be balanced.

[0036] For a certain minority class C i The formula for calculating the number of generated samples is as follows:

[0037]

[0038] in M represents the number of samples generated. i This represents the number of samples in the i-th class, i.e., the sample class C. i The number, M max The maximum number of samples is the dataset D. cbn The number of samples in the class with the most samples, i.e. The number of samples in each category. Since the maximum learning rate is 1, M is used to ensure a balanced number of generated samples. max This serves as the upper limit for the number of samples generated.

[0039] For a certain minority class C i The formula for generating sample points is as follows:

[0040] x new =x p+rand(0,1)×(x q -x p )

[0041] Where, x new It is synthesized new sample data, x p For the minority cluster C i The coordinate vectors of the center point, each minority class has only one center point, p and C. i Correspondingly, p+1 and C i +1 corresponds to x q For the minority cluster C i The coordinate vector of a random sample point in the dataset. p is a natural number. q is a range from 1 to |C|. i A random integer in |C, i | Represents cluster C i The total number of samples is generated using this formula. Each sample point is selected, and the newly generated sample points are added to the minority class C. i In the middle, at this time C i Complete sample balancing;

[0042] Apply the above process to dataset D cbn For each minority sample class, complete dataset D cbn The overall sample balancing yields a new dataset D. nrb .

[0043] S6. Using dataset D nrb The neural network model is retrained, and then updated into the malicious traffic detection module via the adaptive detection upgrade module within the adaptive traffic detection system. At this point, the adaptive traffic detection system's automated update is complete, enabling it to detect unknown threat traffic within the local area network (the detection algorithm is now integrated into the malicious traffic detection module).

[0044] This invention integrates deceptive defense technology and malicious traffic detection technology. It captures unknown threats through deceptive defense technology and expands the dataset to include unknown threats with a small sample size using an improved dataset balancing algorithm. This enables the malicious traffic identification algorithm to identify unknown threats and adaptively expand the attack methods it can identify, further improving the versatility of the malicious traffic detection system. Attached Figure Description

[0045] Figure 1 Deployment diagram of the unknown threat adaptive traffic detection system based on deception defense technology constructed in this invention;

[0046] Figure 2 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0047] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0048] The adaptive malicious traffic detection method of the present invention is based on honeypots. While protecting the security of the industrial control network itself, it uses the attack data in the honeypot to improve the accuracy and versatility of its own identification, enabling the malicious traffic detection module to adaptively detect and discover new network attacks.

[0049] This invention proposes an adaptive detection method for unknown threats based on deception-based defense, implementing the industrial control network deployment described in this method as follows: Figure 1 As shown. In an industrial control network (ICS), there are normally functioning ICS devices. Deploying a honeypot within the ICS will not affect normal business processes. The optimal deployment location for the malicious traffic detection module is the backbone of the ICS, specifically the gateway server.

[0050] like Figure 2 As shown, this adaptive detection method for unknown threats based on deception defense includes the following steps:

[0051] Step 1: Analyze the threat traffic captured by the honeypot using a malicious traffic detection system to obtain unknown traffic. Create a preprocessed dataset D from the analyzed unknown traffic. pre .

[0052] In this embodiment, the threat types in the original dataset include Bot, Brute Force-Web, SQL Injection, and DoS attacks-GoldenEye. This embodiment uses a portion of the CIC-IDS-2017 dataset, with Friday-WorkingHours-Morning.pcap_ISCX.csv serving as the original dataset for this experiment. org .

[0053] Each column in this file represents a session stream. 82 features are selected from these features, some of which are shown in Table 1 below:

[0054] Table 1. Centralized features of the CIC-IDS-2017 malicious network traffic dataset.

[0055] Feature Name illustrate Dst Port Destination port Protocol Protocol type Flow Duration Duration of the flow Tot Fwd Pkts Total number of forward packets Tot Bwd Pkts Total number of reverse packets TotLen Fwd Pkts Total length of forward data packets TotLen Bwd Pkts Total length of reverse data packets Fwd Pkt Len Max Maximum length of forward data packets Fwd Pkt Len Min Minimum length of forward data packets Fwd Pkt Len Mean Average length of forward data packets

[0056] The initial order of some features in the CIC-IDS-2017 malicious network traffic dataset can be set to the order from top to bottom in the table above (the table above shows a partial display of all features), represented as Dst Port, Protocol, Flow Duration, Tot Fwd Pkts, Tot Bwd Pkts, TotLen Fwd Pkts, TotLen Bwd Pkts, Fwd Pkt Len Max, FwdPkt Len Min, Fwd Pkt Len Mean, ...

[0057] Attack traffic of the FTP-BruteForce and SSH-BruteForce types is deployed into the industrial control network. Once deployed, the honeypot module collects this attack traffic (honeypot collection begins immediately after deployment) and feeds it back to the adaptive traffic detection system. The malicious traffic detection module in the adaptive traffic detection system uses a convolutional neural network model to detect the attack traffic returned by the honeypot. If any of the returned attack traffic is detected as legitimate, it is extracted and marked as unknown, creating a preprocessed dataset D. pre .

[0058] Step 2: Preprocess the dataset D pre and the original dataset D org The merged dataset D is obtained. cbn * For the merged dataset D cbn * Clustering is performed on the sample space to obtain the clustered dataset D. cbn And determine the minority and majority classes in the dataset.

[0059] By merging the clusters, two minority class clusters and four majority class clusters were found.

[0060] Step 3: Perform clustering on the dataset D cbn The average distance d between the nearest minority classes in the sample space sa The number N of nearest neighbor majority classes ma ld is a measure of the learning complexity of the minority class samples.

[0061] The weighting coefficient α is set to 0.6. The learning complexities for the two minority clusters are 0.9652 and 0.86521, respectively.

[0062] Step 4: Perform clustering on the dataset D cbn Based on the learning complexity ld, calculate the number of minority class samples synthesized. Generate new minority class samples using the synthesis formula, and merge the generated minority class samples into D. cbnThis forms a new sample dataset D. nrb .

[0063] In this embodiment, the number of Bot class samples is the largest, so the maximum number of samples M is... max It is 16200. D cbn The first minority cluster has 102 samples, and the second minority cluster has 308 samples. Therefore, the number of samples needed to generate for the two minority clusters are 15537 and 13749 respectively. The generated samples are then compared with D... cbn A new sample dataset D is obtained. nrb .

[0064] Step 5: Use dataset D nrb The convolutional neural network model is retrained, and then updated into the malicious traffic detection module through the adaptive detection upgrade module of the adaptive traffic detection system. At this point, the adaptive traffic detection system's automated update is complete, enabling it to detect unknown threat traffic within the local area network.

[0065] The adaptive traffic detection system can detect FTP-BruteForce and SSH-BruteForce threat traffic.

[0066] From the above description, those skilled in the art can clearly understand the implementation method of this method, which can be implemented using the general-purpose hardware platform required by the system. Based on this understanding, those skilled in the art can use a suitable development environment to implement this invention. During the implementation process, the recurrent neural network can be selected according to the actual model requirements, and the type of honeypot needs to be adjusted according to the actual scenario.

[0067] In summary, the method of this invention can further improve the accuracy of existing machine learning models. The embodiments described above are merely some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

Claims

1. An adaptive detection method for unknown threats based on deception defense, characterized in that, This method employs an adaptive traffic detection system for unknown threats based on deception defense technology. The system is deployed in an existing local area network as follows: industrial control equipment, routers, and switches are existing devices; the gateway server and honeypot devices are equipped with the adaptive traffic detection system. The malicious traffic detection module and adaptive detection upgrade module within the adaptive traffic detection system are deployed in the gateway server. Each honeypot device is equipped with a honeypot module of the adaptive traffic detection system. The honeypot module has deception defense capabilities, used to guide and capture unknown threat traffic targeting the honeypot device, and feeds back the captured unknown threat traffic to the gateway server. One or more honeypot devices are deployed in the industrial control network. The malicious traffic detection module in the adaptive traffic detection system performs threat detection using an existing neural network model, and the adaptive detection upgrade module upgrades and updates the malicious traffic detection module by retraining the neural network model. The method includes the following steps: S1. Analyze the threat traffic captured by the honeypot module using the malicious traffic detection module to obtain unknown threat traffic, and create a preprocessed dataset from the unknown threat traffic. Meanwhile, the dataset initially carried by the malicious traffic detection module is recorded as the original dataset. ; S2, Preprocess the dataset and the original dataset Merge the datasets to obtain the merged dataset. For merging datasets Clustering is performed on the sample space to obtain the clustered dataset. and determine the dataset The division of minority and majority classes in the text; The following is about merging datasets. Clustering formula and clustered dataset process: Before performing clustering operations, It consists of two parts and ; Central Plains belong The sample points have a complete set of cluster identifiers. , ,in, Indicates the first Class center point location, The total number of categories in the original dataset is , Central Plains belong Each sample point is treated as an independent cluster; this is accomplished using the following formula. Clustering and obtaining New cluster identifier set The following formula is used for calculation. If the distance between all clusters is less than a preset threshold, If so, the two clusters will be merged; The formula for calculating cluster distance is as follows: in, and Let x and y represent two arbitrary clusters, respectively. and The sample points in and This represents the total number of samples in the cluster. Less than Then, the two clusters are merged, and this process is repeated until no further merging is possible. The resulting clusters are then sorted from largest to smallest according to the number of samples. The new clusters have This yields the clustered dataset. ; The calculation process shows that the new sample points are aggregated into two places: the first is the existing ones. Of the three categories, the second one forms a new category. Due to the small number of unknown threat samples, the first... The known categories are the majority class, and then... - These categories are few, with unknown threats concentrated in a few categories; S3. The clustered dataset By the average distance of the nearest minority class in the sample space Number of nearest neighbor majority classes Measure the learning complexity of the minority class samples. ; For a certain minority class The calculation process for the average distance between neighboring minority classes is as follows, where, For set For a certain class in the set, the range of values ​​for i is... +1 to Integers: Average distance to m neighboring minority classes ,for The average distance between neighboring minority classes represents the sparsity of samples of that class in the minority class space, where m is a pre-set constant, and the m neighboring minority classes represent... The top m minority classes, sorted by distance from all minority classes in ascending order; For a certain minority class The calculation process for the number of local nearest neighbor majority classes is as follows: The circle formed by the distance to the m-th nearest neighbor (minority class) has the following radius: The number of majority class samples within this circle is... ,express The number of local nearest neighbor majority classes; Average distance of neighboring minority classes The larger the value, the sparser the spatial distribution of minority class samples, making them difficult to learn and thus requiring higher learning weights; conversely, the denser the distribution, the easier it is to learn and the lower the learning complexity. Number of local nearest neighbor majority classes The larger the value, the more likely the sample is located on the boundary or in the center of the majority sample class. In this case, the learning cost of the sample will be very high because the sample will be affected by the majority sample class. Conversely, the lower the value, the lower the learning complexity and the easier it is to learn. The formula for normalizing the mean distance between nearest neighbors and minority classes is as follows: Where j takes the value of +1 to positive integers, This represents the average distance between the j-th minority class and its nearest neighboring minority classes. This represents the normalized value of the average distance between the nearest minority classes of the i-th minority class; The formula for normalizing the number of local nearest neighbor majority classes is as follows: This represents the number of local nearest neighbor majority classes. This represents the normalized value of the number of the i-th local nearest neighbor majority class; For minority classes learning complexity The calculation formula is as follows: in The weighting coefficients represent the relationship between the average distance of the nearest minority class and the number of local nearest majority classes; the learning complexity is... The larger the value, the higher the value of the sample class. The more difficult it is to learn, the more neighboring samples need to be generated; S4. Analyze the clustered dataset. According to learning complexity Calculate the number of minority class samples synthesized, generate new minority class samples using the synthesis formula, and merge the generated minority class samples into the new minority class samples. To form a new sample dataset Dataset The sample space tends to be balanced; For a certain minority class The formula for calculating the number of generated samples is as follows: in The number of samples generated. This represents the number of samples in the i-th class, i.e., the number of sample classes. The number, The maximum number of samples is the dataset. The number of samples in the class with the most samples, i.e. Number of samples in each category; For a certain minority class The formula for generating sample points is as follows: in, It is synthesized new sample data. For a minority of clusters The coordinate vector of the center point, each minority class has only one center point. For a minority of clusters A random sample point coordinate vector, where p is a natural number and q is a number from 1 to q. random integers in Representing a cluster The total number of samples is generated using this formula. Each sample point is selected, and the newly generated sample points are added to the minority class. In the middle, at this time Complete sample balancing; Apply the above process to the dataset Complete the dataset for each minority sample class. Overall sample balancing yields a new dataset. ; S5. Using the dataset The neural network model is retrained, and then updated into the malicious traffic detection module through the adaptive detection upgrade module in the adaptive traffic detection system. At this point, the adaptive traffic detection system has completed its automatic update and is able to detect unknown threat traffic in the local area network.

2. The adaptive detection method for unknown threats based on deception defense according to claim 1, characterized in that, The original dataset The types of threats include Bot, Brute Force - Web, SQL Injection, and DoSattacks - GoldenEye.

3. The adaptive detection method for unknown threats based on deception defense according to claim 2, characterized in that, In step S3 The value is 0.6.

Images