An internet of things intrusion detection method and system

Abstract

The application discloses an Internet of Things intrusion detection method and system, and relates to the fields of information security and Internet of Things; the method comprises the following steps: carrying out feature selection, one-hot encoding, normalization, sample balancing and data cleaning preprocessing on Internet of Things traffic data; training a generative deep learning model AAE and BiGAN based on the preprocessed IoT-23 data set; after the real-time collected traffic is preprocessed, the AAE or BiGAN model is used to distinguish abnormal and normal traffic, and then a KNN classifier is used to determine the attack type; the abnormal traffic features are stored in an IoTDB time series database; according to the review results of a vulnerability library and a threat intelligence library, the TPR, TNR, FPR, FNR and unknown category proportion UNR are calculated, the AAE and BiGAN are switched according to the UNR, the model retraining is triggered according to the TPR-FPR difference or the TNR-FNR difference, and the sample balancing coefficient R is dynamically adjusted. Through the two-stage detection framework and the dynamic feedback mechanism, the application improves the Internet of Things intrusion detection accuracy and reduces the deployment cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information security and Internet of Things, and particularly relates to an Internet of Things intrusion detection method and system. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and can not constitute the prior art.

[0003] Statistical data shows that the number of global networked devices has grown rapidly in the past decade. According to this trend, the number of networked Internet of Things devices may exceed 25 billion by 2030. This unprecedented scale of device interconnection provides a broad development space for the digital transformation of various industries, but the Internet of Things technology also brings new security risks while providing convenience. Most Internet of Things devices have limited computing power, difficulty in system upgrading, and insufficient security protection, and often become a breakthrough for attacks. Improving the security, stability and reliability of the Internet of Things system has become a key research topic in the field of network security.

[0004] In the aspect of Internet of Things intrusion detection, existing researches adopt machine learning or deep learning methods, which are prone to majority class overfitting due to highly unbalanced data (normal traffic usually accounts for more than 90%). Moreover, the Internet of Things environment has the characteristic of emerging new attacks, and the detection accuracy of existing methods for common attacks, especially unknown attacks, is not high enough.

[0005] Existing intrusion detection methods mostly adopt simple single-level detection mode, and the analysis of intrusion detection results is not detailed enough. Most existing researches, the model output is usually limited to binary classification of whether it is abnormal or not, and rarely gives abnormal alarm and attack type distribution combined with detection data. In practical applications, this cannot help security personnel or administrators determine the specific type and distribution of attacks, so as to locate the attack source faster and take appropriate defense measures.

[0006] The existing intrusion detection method has a single sample balancing method, which leads to a low detection accuracy for low-frequency attacks. Existing researches mostly directly use SMOTE or ADASYN oversampling techniques as the sample balancing method, but this may bring two problems: one is to increase the interference of false data, and the other is to make the model overfit. The number of samples of different types of attacks in the Internet of Things often differs greatly, and the samples are very unbalanced. If the detection model only relies on a large number of standard data for training, it is difficult to accurately identify new or rare attack methods. SUMMARY

[0007] The application aims at the problems in the prior art, and provides an Internet of Things intrusion detection method and system, a secondary detection framework of a fusion generative deep learning model AAE / BiGAN+KNN, a mixed sample balancing strategy is adopted to further improve the detection accuracy; specifically, the application adopts a mixed sample balancing strategy: as described in step S4, compress the maximum class samples to R% of the original number to obtain K samples, and oversample the small class samples with a sample number less than K*N% to K*N% by SMOTE. In this way, by compressing the maximum class samples and oversampling the minimum class, the class imbalance is significantly alleviated, and the interference of false data and model overfitting caused by directly using oversampling technologies such as SMOTE or ADASYN is reduced.

[0008] The technical scheme of the application is as follows:

[0009] An Internet of Things intrusion detection method, comprising:

[0010] Step S1: feature selection; extracting features with high discrimination and prediction ability for intrusion detection from Internet of Things traffic data;

[0011] Step S2: feature encoding; one-hot encoding of the category type features in the features;

[0012] Step S3: normalization; minimum-maximum normalization of the numerical type features in the features;

[0013] Step S4: sample balancing; compressing the maximum class samples to R% of the original number to obtain K samples, and oversampling the small class samples with a sample number less than K*N% to K*N% by SMOTE;

[0014] Step S5: data cleaning; deleting null records and filling missing values with feature average values;

[0015] Step S6: based on the IoT23 dataset, generating a historical dataset after data preprocessing according to steps S1-S5; and training a generative deep learning model based on the historical dataset; the generative deep learning model comprises an AAE model and a BiGAN model;

[0016] Step S7: real-time acquisition of Internet of Things traffic data and execution of steps S1-S5 for preprocessing to obtain a real-time data stream;

[0017] Step S8: using the generative deep learning model to perform intrusion detection on the real-time data stream, and using the AAE model or the BiGAN model to distinguish possible abnormal traffic and normal traffic;

[0018] Step S9: using a KNN classifier to judge the attack type of the abnormal traffic identified in step S8.

[0019] Step S10: The determination results of each of steps S8 and S9 are used to cache the characteristics of abnormal traffic by the IoTDB Internet of Things time series database;

[0020] Step S11: At every preset time, the determination results of all traffic since the interval time are reviewed according to the vulnerability library and the threat intelligence library, the correct positive rate TPR, the correct negative rate TNR, the false positive rate FPR, the false negative rate FNR and the unknown category proportion UNR are calculated according to the actual review results, the unknown category proportion UNR is used to switch the generated deep learning model, the difference between the correct positive rate TPR and the false positive rate FPR or the difference between the correct negative rate and the false negative rate FNR is used to trigger the retraining of the corresponding model, and the R value is dynamically adjusted.

[0021] Further, the characteristics with high discrimination and prediction ability for intrusion detection are determined by the following method:

[0022] By calculating the correlation between each feature, the features with high contribution to classification results are retained.

[0023] Further, the characteristics with high discrimination and prediction ability for intrusion detection include:

[0024] Network protocol, service type, connection duration, source device transmission byte number, target device transmission byte number, connection state, source device transmission packet number and target device transmission packet number.

[0025] Further, the AAE model includes:

[0026] The encoder is responsible for compressing the 27-dimensional input features into a 6-dimensional latent space;

[0027] The generator is responsible for restoring the original data based on the latent space;

[0028] The discriminator is used to calculate the adversarial loss, distinguish between real data and generated data, and continuously update the parameters of the encoder and the generator to form a dynamic adversarial game process;

[0029] The loss function of the AAE model is divided into two parts, one is to use mean square error as the autoencoder loss to quantify the deviation degree of the generated data and the original data, and the other is to introduce binary cross entropy as the adversarial loss to improve the authenticity of the generated data through the adversarial training of the generator and the discriminator.

[0030] Further, the BiGAN model includes:

[0031] The encoder is responsible for converting the original input data into an 8-dimensional latent feature representation;

[0032] a generator capable of synthesizing network traffic data according to a random noise vector in a latent space;

[0033] a discriminator capable of receiving two sets of contrast samples simultaneously and identifying data authenticity through comparative analysis, wherein the first set of contrast samples comprises original input data and encoder output, and the second set of contrast samples comprises data generated by the generator and noise input.

[0034] Further, the step S8 comprises:

[0035] The AAE model is used by default to distinguish possible abnormal traffic and normal traffic.

[0036] Further, the step S11 of switching the generative deep learning model according to the unknown class ratio UNR comprises:

[0037] When the unknown class ratio UNR exceeds the threshold X, the BiGAN model is switched to be used to distinguish possible abnormal traffic and normal traffic, and when the unknown class ratio UNR is lower than the threshold X, the AAE model is switched.

[0038] Further, the step S11 of retraining comprises:

[0039] For the AAE model, when the difference between the correct positive rate TPR and the false positive rate FPR is less than the threshold Y, the abnormal traffic recorded since the interval time is added to the sample data after being adjusted according to the review result to start the AAE model for retraining.

[0040] For the BiGAN model, when the difference between the correct negative rate TNR and the false negative rate FNR is less than the threshold Z, the abnormal traffic recorded since the interval time is added to the sample data after being adjusted according to the review result to start the BiGAN model for retraining.

[0041] Further, the step S11 of dynamically adjusting the R value comprises:

[0042] For the AAE model, when the difference between the correct positive rate TPR and the false positive rate FPR is less than the threshold T, the R value is increased.

[0043] When the difference between the correct positive rate TPR and the false positive rate FPR is greater than the threshold T, the R value is decreased.

[0044] For the BiGAN model, when the difference between the correct negative rate TNR and the false negative rate FNR is less than the threshold M, the R value is increased.

[0045] When the difference between the correct negative rate TNR and the false negative rate FNR is greater than the threshold M, the R value is decreased.

[0046] This invention also proposes an Internet of Things (IoT) intrusion detection system, comprising:

[0047] The data preprocessing module is used to preprocess IoT traffic data; preprocessing includes: feature selection, feature encoding, sample balancing, and data cleaning.

[0048] Generative deep learning models include: AAE model and BiGAN model; the AAE model learns the latent feature distribution of normal samples through an encoder-generator structure, and the BiGAN model obtains the ability to generate and discriminate new samples through adversarial training between the generator and the encoder.

[0049] The core detection and alarm module uses a two-level detection framework for traffic intrusion detection. The first level of detection uses the AAE model or BiGAN model to identify possible abnormal traffic; the second level uses the KNN classifier to determine the attack type.

[0050] The results review and feedback module caches traffic features for each judgment result and reviews the judgment results of all traffic during that period at regular intervals. Based on the actual review results, it calculates the correct affirmative rate (TPR), correct negative rate (TNR), false affirmative rate (FPR), false negative rate (FNR), and unknown category ratio (UNR) of the traffic. It also adjusts the model used for the first-level detection based on the unknown category ratio (UNR) and adjusts the training samples of the generative deep learning model based on the correct affirmative rate (TPR), correct negative rate (TNR), false affirmative rate (FPR), and false negative rate (FNR).

[0051] Compared with existing technologies, the advantages of this invention are:

[0052] 1. This invention significantly alleviates class imbalance and improves the model's detection sensitivity for a few attack categories by using maximum class sample compression and minimum class oversampling.

[0053] 2. This invention adopts a two-level unsupervised framework of AAE, BiGAN and KNN to achieve abnormal traffic detection and fine classification without manual annotation, reducing deployment costs and adapting to the real-time needs of IoT scenarios.

[0054] 3. By dynamically adjusting the R value, this invention can balance detection accuracy and computational overhead online while maintaining the overall data scale, thus ensuring stable performance of the system under different network loads.

[0055] 4. This invention proposes a two-level detection framework that integrates generative deep learning models AAE / BiGAN+KNN and adopts a hybrid sample balancing strategy to further improve detection accuracy. Attached Figure Description

[0056] Figure 1 This is a flowchart of an IoT intrusion detection method. Detailed Implementation

[0057] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] For ease of understanding, the technical terms mentioned in this invention are explained as follows:

[0059] An intrusion detection system (IDS) is a network security device that monitors network transmissions in real time, issuing alerts or taking proactive measures when suspicious transmissions are detected. What distinguishes it from other network security devices is that IDS is a proactive security protection technology.

[0060] The Internet of Things (IoT) originated in the media field and represents the third revolution in the information technology industry. IoT refers to connecting any object to a network through information sensing devices and according to agreed-upon protocols. These objects exchange and communicate information through information transmission media to achieve functions such as intelligent identification, location, tracking, and monitoring.

[0061] AAE (Adversarial Autoencoder) is a hybrid generative model that combines autoencoders and generative adversarial networks (GANs). It improves the quality of generation by adversarial training that constrains the distribution of latent variables.

[0062] BiGAN (Bidirectional Generative Adversarial Network) is a deep learning model that combines an encoder with a generative adversarial network (GAN) to improve the quality of generated images through a bidirectional learning process.

[0063] The KNN (K-Nearest Neighbors) algorithm is an instance-based supervised learning classification algorithm that determines the category of a sample to be classified by calculating the distance between the sample and its K nearest neighbors in the training set and based on the majority voting principle.

[0064] SMOTE (Synthetic Minority Oversampling Technique) is an improvement on the random oversampling method. Random oversampling increases the number of minority samples by simply copying existing samples, which can easily lead to model overfitting. The basic idea of ​​SMOTE is to analyze the minority class samples and artificially synthesize new samples.

[0065] IoT-23: The IoT-23 dataset consists of 23 network captures of IoT traffic (called scenarios), including 20 from infected IoT devices (PCAP files) and three from real IoT network traffic.

[0066] Time series database: A database system specifically designed for processing and storing time-series data. This data type has a defined time order and is typically used to track and analyze the state or measurements of various entities at a specific point in time.

[0067] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0068] Example 1

[0069] An IoT intrusion detection method includes four stages: data preprocessing stage, generative deep learning model training stage, core detection and alarm stage, and result review and feedback stage.

[0070] In this embodiment, specifically, an IoT intrusion detection method includes the following steps:

[0071] Step S1: Feature selection; Extract features with high discriminative and predictive capabilities for intrusion detection from IoT traffic data;

[0072] In this embodiment, the features with high discriminative and predictive capabilities for intrusion detection are determined using the following method:

[0073] By calculating the correlation between features, features that contribute highly to the classification results are retained. It should be noted that the Pearson correlation coefficients between each feature are calculated as follows:

[0074]

[0075] in:

[0076] Representation of features With features The covariance between the two variables measures whether their linear trends are consistent.

[0077] Representation of features Standard deviation, a measure The degree of dispersion of its own data;

[0078] Representation of features Standard deviation, a measure The degree of dispersion of its own data;

[0079] Representation of features With features The Pearson correlation coefficient between them ranges from [-1, 1].

[0080] By calculating the correlation between features, those features that contribute significantly to the classification results are retained. Through data analysis, the following features are retained:

[0081] Network protocol, service type, connection duration, number of bytes transferred by the source device, number of bytes transferred by the destination device, connection status, number of data packets transferred by the source device, and number of data packets transferred by the destination device;

[0082] Among them are network protocols such as TCP, UDP, and ICMP;

[0083] Service types include HTTP, DNS, FTP, etc.

[0084] Step S2: Feature encoding; perform one-hot encoding on the categorical features;

[0085] This involves converting categorical data into a numerical format that the model can process. Since deep learning models cannot directly handle non-numerical inputs, categorical features must be encoded. Network protocols, service types, and connection states are typical categorical variables, and one-hot encoding is used to quantify them, mapping each category to a mutually exclusive binary vector.

[0086] Step S3: Normalization; Perform min-max normalization on the numerical features;

[0087] In this embodiment, it should be noted that the numerical features in the original data have orders of magnitude differences. Therefore, all numerical features (such as connection duration, number of bytes transmitted by the source device, number of bytes transmitted by the target device, etc.) are normalized to scale the values ​​of each feature to a uniform range, thereby improving the training efficiency and accuracy of the model. This invention uses min-max normalization for numerical features such as connection duration, number of bytes transmitted by the source device, and number of bytes transmitted by the target device, limiting their range to [0, 1].

[0088]

[0089] in:

[0090] This represents the normalization result;

[0091] Indicates the maximum value;

[0092] Indicates the minimum value;

[0093] This represents the value that needs to be normalized.

[0094] Step S4: Sample balancing; Compress the largest class samples to R% (default 25%) of their original number to obtain K samples, and at the same time, oversample the smaller class samples with a sample number less than K×N% (default 10%) to K×N% using SMOTE;

[0095] For example, the samples of the largest class are compressed to 25% of their original number, resulting in K samples. At the same time, the samples of the smaller classes with a sample size of less than K×10% are oversampled by SMOTE to K×10%. By controlling the ratio of the largest class to the smallest class to not exceed 10:1, the sample bias problem can be alleviated.

[0096] Step S5: Data cleaning; delete null records and fill missing values ​​with the feature mean;

[0097] In this embodiment, it should be noted that there are some null and duplicate values ​​in the original data, which interfere with model training. Therefore, data cleaning is required. During the data cleaning process, null records in the dataset are deleted, especially those records with null values ​​in features such as connection duration, number of bytes transmitted from the source device, and number of bytes transmitted from the target device. The missing values ​​are filled with the average value of the features.

[0098] Step S6: Based on the IoT23 dataset, generate a historical dataset after preprocessing the data according to steps S1-S5; and train a generative deep learning model based on the historical dataset; the generative deep learning model includes: AAE model and BiGAN model;

[0099] In this embodiment, specifically, the AAE model includes:

[0100] The encoder is responsible for compressing the 27-dimensional input features into a 6-dimensional latent space;

[0101] The generator is responsible for reconstructing the original data based on the potential space;

[0102] The discriminator is used to calculate the adversarial loss, distinguish between real data and generated data, and continuously update the parameters of the encoder and generator to form a dynamic adversarial game process.

[0103] The loss function of the AAE model is divided into two parts. The first part uses mean squared error (MSE) as the autoencoder loss to quantify the degree of deviation between the generated data and the original data. The second part introduces binary cross-entropy (BCE) as adversarial loss to improve the authenticity of the generated data through adversarial training between the generator and the discriminator.

[0104] In this embodiment, specifically, the BiGAN model includes:

[0105] The encoder is responsible for converting the raw input data into an 8-dimensional latent feature representation;

[0106] The generator is capable of synthesizing network traffic data based on random noise vectors in the latent space;

[0107] The discriminator can simultaneously receive two sets of comparison samples and identify the authenticity of the data through comparative analysis. The first set of comparison samples includes the original input data and the encoder output, while the second set of comparison samples includes the data generated by the generator and the noise input.

[0108] Step S7: Collect IoT traffic data in real time and perform preprocessing steps S1-S5 to obtain a real-time data stream;

[0109] This involves collecting raw data in real time from various IoT devices and performing S1-S5 preprocessing operations on the collected data to build a unified and standardized real-time data stream.

[0110] Step S8: Use a generative deep learning model to perform intrusion detection on the real-time data stream, and use the AAE model or BiGAN model to distinguish between possible abnormal traffic and normal traffic;

[0111] In this embodiment, it should be noted that the AAE model is used by default to distinguish between possible abnormal traffic and normal traffic.

[0112] Step S9: Use the KNN classifier to determine the attack type for the abnormal traffic identified in step S8;

[0113] In this embodiment, it should also be noted that the detection results of steps S8 and S9 can also be presented in the form of charts and alarm information.

[0114] Step S10: For each determination result in steps S8 and S9, use the IoTDB Internet of Things time-series database to cache the characteristics of abnormal traffic;

[0115] Step S11: At preset intervals, review the judgment results of all traffic since the interval based on the vulnerability database and threat intelligence database. Calculate the True Positive Rate (TPR), True Negative Rate (TNR), False Positive Rate (FPR), False Negative Rate (FNR), and Unknown Category Ratio (UNR) based on the actual review results. Switch the generative deep learning model according to the Unknown Category Ratio (UNR). Trigger retraining of the corresponding model based on the difference between the True Positive Rate (TPR) and the False Positive Rate (FPR) or the difference between the True Negative Rate (TPR) and the False Negative Rate (FNR), while dynamically adjusting the R value.

[0116] In this embodiment, it should be noted that the vulnerability database includes major domestic and international vulnerability databases such as CVE, CNVD, CNNVD, and AVD, and the threat intelligence database includes major domestic and international threat intelligence databases such as OTX.

[0117] In this embodiment, it should be noted that the True Positive Rate (TPR) is the ratio of a predicted value to an actual value that is true.

[0118] True Negative Rate (TNR): The rate at which a prediction is false and the actual result is false.

[0119] False Positive Rate (FPR): The rate at which a prediction is true but the actual result is false.

[0120] False Negative Rate (FNR): The rate at which a prediction is false but the actual result is true.

[0121] In this embodiment, it should be noted that the correctness rate (TPR) is the ratio of traffic identified as normal in step S8 and reviewed as normal in step S11 to traffic identified as normal in step S8.

[0122] The correct rejection rate (TNR) is the ratio of traffic identified as abnormal in step S8 and reviewed as abnormal in step S11 to traffic identified as abnormal in step S8.

[0123] Error probability ratio (FPR) is the ratio of traffic identified as normal in step S8 but examined as abnormal in step S11 to traffic identified as normal in step S8.

[0124] Error rejection rate (FNR) is the ratio of traffic identified as abnormal in step S8 but reviewed as normal in step S11 to traffic identified as abnormal in step S8.

[0125] In this embodiment, specifically, step S11, which involves switching the generative deep learning model based on the Unknown Class Ratio (UNR), includes:

[0126] When the proportion of unknown categories (UNR) exceeds the threshold X, the system switches to the BiGAN model to distinguish between possible abnormal traffic and normal traffic; when the proportion of unknown categories (UNR) is below the threshold X, the system switches to the AAE model.

[0127] In this embodiment, it should also be noted that the Unknown Category Ratio (UNR) is the ratio of the abnormal traffic identified in step S8 in step S9, after the attack type is determined using the KNN classifier, to the abnormal traffic whose attack type cannot be determined, and the abnormal traffic identified in step S8.

[0128] In this embodiment, specifically, the retraining in step S11 includes:

[0129] For the AAE model, when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is less than the threshold Y, the abnormal traffic recorded since the interval time is adjusted according to the review results and added to the sample data to start the AAE model for retraining.

[0130] Specifically, firstly, the portion of abnormal traffic recorded over the interval period that also shows abnormal traffic results is added to the sample data (the basic sample data is based on the IoT23 dataset, and new abnormal traffic data is added sequentially to expand the sample), expanding it into new sample data; further, after preprocessing the sample data as described in steps S1-S5, a new historical dataset is formed; then, the encoder (responsible for compressing the 27-dimensional input features into a 6-dimensional latent space), the generator (responsible for restoring the original data based on the latent space), and the discriminator (used to calculate the adversarial loss and distinguish between real data and generated data) are run accordingly, and the parameters of the encoder and generator are continuously updated;

[0131] For the BiGAN model, when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is less than the threshold Z, the abnormal traffic recorded since the interval time is adjusted according to the review results and added to the sample data to start the BiGAN model for retraining.

[0132] Specifically, firstly, the portion of abnormal traffic recorded over the interval period that also shows abnormal traffic results is added to the sample data (the basic sample data is based on the IoT23 dataset, and new abnormal traffic data is added sequentially to expand the sample), expanding it into new sample data; further, after preprocessing the sample data as described in steps S1-S5, a new historical dataset is formed; then, the encoder (responsible for converting the original input data into an 8-dimensional latent feature representation), the generator (synthesizing network traffic data based on random noise vectors in the latent space), and the discriminator (simultaneously receiving two sets of comparison samples and identifying the authenticity of the data through comparative analysis) are run, and the parameters of the encoder and generator are continuously updated.

[0133] In this embodiment, specifically, the dynamic adjustment of the R value in step S11 includes:

[0134] For the AAE model, when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is less than the threshold T, the R value should be increased appropriately; when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is greater than the threshold T, the R value should be decreased appropriately.

[0135] For the BiGAN model, when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is less than the threshold M, the R value should be increased appropriately; when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is greater than the threshold M, the R value should be decreased appropriately.

[0136] In this embodiment, an R-value adjustment algorithm is further proposed, as follows:

[0137] For the AAE model, when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is less than the threshold T, the R value is increased. The specific adjustment method for the R value is as follows:

[0138] ;

[0139] When the difference between the correct affirmation rate (TPR) and the incorrect affirmation rate (FPR) is greater than the threshold T, the R value is reduced. The specific adjustment method for the R value is as follows:

[0140] ;

[0141] For the BiGAN model, when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is less than the threshold M, the R value is increased. The specific adjustment method for the R value is as follows:

[0142] ;

[0143] When the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is greater than the threshold M, the R value is reduced. The specific adjustment method for the R value is as follows:

[0144] .

[0145] In this embodiment, specifically, an IoT intrusion detection system is proposed to implement the above-mentioned IoT intrusion detection method, which specifically includes the following modules:

[0146] The data preprocessing module is used to preprocess IoT traffic data; preprocessing includes: feature selection, feature encoding, sample balancing, and data cleaning.

[0147] Generative deep learning models include: AAE model and BiGAN model; the AAE model learns the latent feature distribution of normal samples through an encoder-generator structure, and the BiGAN model obtains the ability to generate and discriminate new samples through adversarial training between the generator and the encoder.

[0148] The core detection and alarm module employs a two-level detection framework for traffic intrusion detection. The first level uses an AAE model or a BiGAN model to identify potential abnormal traffic; the second level uses a KNN classifier to determine the attack type; and KNN is used as a post-processing model to determine the specific attack type, thus enabling the identification and response to IoT intrusion behaviors. When the system detects an anomaly, it presents the detection results in the form of charts and alarm information.

[0149] The results review and feedback module caches traffic features for each judgment result and reviews the judgment results of all traffic during that period at regular intervals. Based on the actual review results, it calculates the correct affirmative rate (TPR), correct negative rate (TNR), false affirmative rate (FPR), false negative rate (FNR), and unknown category ratio (UNR) of the traffic. It also adjusts the model used for the first-level detection based on the unknown category ratio (UNR) and adjusts the training samples of the generative deep learning model based on the correct affirmative rate (TPR), correct negative rate (TNR), false affirmative rate (FPR), and false negative rate (FNR).

[0150] In this embodiment, it should be noted that the system proposed in this embodiment adopts a two-level detection framework for traffic intrusion detection. The first level of detection uses AAE or BiGAN models to identify possible abnormal traffic, while the second level uses a KNN classifier to determine the attack type of suspicious traffic.

[0151] In this embodiment, it should also be noted that the first-level detection switches between the AAE model and the BiGAN model based on the calculation results of the proportion of unknown categories detected in traffic at regular intervals (the AAE model is used by default; when the proportion of unknown categories exceeds a certain threshold, it switches to the BiGAN model; when the proportion of unknown categories is lower than a certain threshold, it switches back to the AAE model). Based on the calculation results of indicators such as TPR, TNR, FPR, and FNR, the traffic cached during this period is added to the extended sample data training sample to start the training module. For the AAE model, when the difference between TPR and FPR is less than a certain threshold, the extended sample data is used to start the AAE model for retraining. For the BiGAN model, when the difference between TNR and FNR is less than a certain threshold, the extended sample data is used to start the BiGAN model for retraining.

[0152] The following is in conjunction with the appendix Figure 1 The above-mentioned IoT intrusion detection method will be further explained.

[0153] Figure 1 The “data preprocessing” in the process involves two aspects: firstly, preprocessing the IoT traffic data obtained from “data stream acquisition” of “IoT devices” to obtain “real-time data streams” for subsequent “abnormal traffic identification”; secondly, preprocessing the “IoT-23 dataset” to obtain “batch dataset data” for “model training” in step S6.

[0154] Figure 1 The "abnormal traffic identification" in the text refers to step S8, which obtains abnormal data streams through abnormal traffic identification.

[0155] Figure 1 The "abnormal traffic classification" in step S9 involves using a KNN classifier to classify abnormal traffic into second-level attack types and "statistical data". Finally, the detection results of steps S8 and S9 are presented in the form of "chart display and alarm".

[0156] Figure 1 The "Result Review and Feedback" step is to execute step S11 and start the retraining of the AAE model or BiGAN model (i.e., "Model Update") based on the "New Sample Data" after the review results are adjusted; at the same time, the R value is dynamically adjusted to achieve "Balanced Parameters"; and it can determine whether "Model Switching" is needed based on the Unknown Class Ratio (UNR).

[0157] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0158] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. An Internet of Things (IoT) intrusion detection method, characterized in that, include: Step S1: Feature selection; Extract features from IoT traffic data that have high discriminative and predictive capabilities for intrusion detection; Step S2: Feature encoding; One-hot encoding is performed on categorical features. Step S3: Normalization; Perform min-max normalization on the numerical features; Step S4: Sample balancing; Compress the largest class samples to R% of their original number to obtain K samples, and at the same time, perform SMOTE oversampling on the smaller class samples with a sample size of less than K×N% to K×N%; Step S5: Data cleaning; delete null records and fill missing values ​​with the feature mean; Step S6: Based on the IoT23 dataset, generate a historical dataset after preprocessing the data according to steps S1-S5; and train a generative deep learning model based on the historical dataset. The generative deep learning models include: AAE model and BiGAN model; Step S7: Collect IoT traffic data in real time and perform preprocessing steps S1-S5 to obtain a real-time data stream; Step S8: Use a generative deep learning model to perform intrusion detection on the real-time data stream, and use the AAE model or BiGAN model to distinguish between possible abnormal traffic and normal traffic; Step S9: Use the KNN classifier to determine the attack type for the abnormal traffic identified in step S8; Step S10: For each determination result in steps S8 and S9, use the IoTDB Internet of Things time-series database to cache the characteristics of abnormal traffic; Step S11: At preset intervals, review the judgment results of all traffic since the interval based on the vulnerability database and threat intelligence database. Calculate the correct affirmative rate (TPR), correct negative rate (TNR), incorrect affirmative rate (FPR), incorrect negative rate (FNR), and unknown category ratio (UNR) based on the actual review results. Switch the generative deep learning model based on the unknown category ratio (UNR). Trigger retraining of the corresponding model based on the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) or the difference between the correct negative rate (TPR) and the incorrect negative rate (FNR), while dynamically adjusting the R value.

2. The IoT intrusion detection method according to claim 1, characterized in that, Features with high discriminative and predictive capabilities for intrusion detection are determined using the following method: By calculating the correlation between features, features that contribute highly to the classification results are retained.

3. The IoT intrusion detection method according to claim 2, characterized in that, Features that enable high discrimination and predictive ability in intrusion detection include: Network protocol, service type, connection duration, number of bytes transferred by the source device, number of bytes transferred by the destination device, connection status, number of data packets transferred by the source device, and number of data packets transferred by the destination device.

4. The IoT intrusion detection method according to claim 1, characterized in that, The AAE model includes: The encoder is responsible for compressing the 27-dimensional input features into a 6-dimensional latent space; The generator is responsible for reconstructing the original data based on the potential space; The discriminator is used to calculate the adversarial loss, distinguish between real data and generated data, and continuously update the parameters of the encoder and generator to form a dynamic adversarial game process. The loss function of the AAE model consists of two parts: first, the mean squared error is used as the autoencoder loss to quantify the degree of deviation between the generated data and the original data; second, binary cross-entropy is introduced as an adversarial loss to improve the authenticity of the generated data through adversarial training between the generator and the discriminator.

5. The IoT intrusion detection method according to claim 1, characterized in that, The BiGAN model includes: The encoder is responsible for converting the raw input data into an 8-dimensional latent feature representation; The generator is capable of synthesizing network traffic data based on random noise vectors in the latent space; The discriminator can simultaneously receive two sets of comparison samples and identify the authenticity of the data through comparative analysis. The first set of comparison samples includes the original input data and the encoder output, while the second set of comparison samples includes the data generated by the generator and the noise input.

6. The IoT intrusion detection method according to claim 1, characterized in that, Step S8 includes: The default setting uses the AAE model to distinguish between possible abnormal traffic and normal traffic.

7. The IoT intrusion detection method according to claim 6, characterized in that, The step S11, which involves switching the generative deep learning model based on the Unknown Class Ratio (UNR), includes: When the proportion of unknown categories (UNR) exceeds the threshold X, the system switches to the BiGAN model to distinguish between possible abnormal traffic and normal traffic. When the proportion of unknown categories (UNR) is below the threshold X, the system switches to the AAE model.

8. The IoT intrusion detection method according to claim 1, characterized in that, The retraining in step S11 includes: For the AAE model, when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is less than the threshold Y, the abnormal traffic recorded since the interval time is adjusted according to the review results and added to the sample data to start the AAE model for retraining. For the BiGAN model, when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is less than the threshold Z, the abnormal traffic recorded since the interval is adjusted according to the review results and added to the sample data to start the BiGAN model for retraining.

9. The IoT intrusion detection method according to claim 1, characterized in that, The dynamic adjustment of the R value in step S11 includes: For the AAE model, when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is less than the threshold T, the R value is increased; when the difference between the correct affirmative rate (TPR) and the incorrect affirmative rate (FPR) is greater than the threshold T, the R value is decreased. For the BiGAN model, the R value is increased when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is less than the threshold M; the R value is decreased when the difference between the correct rejection rate (TNR) and the incorrect rejection rate (FNR) is greater than the threshold M.

10. An Internet of Things (IoT) intrusion detection system, characterized in that, include: The data preprocessing module is used to preprocess IoT traffic data; Preprocessing includes: feature selection, feature encoding, sample balancing, and data cleaning; Generative deep learning models include: AAE model and BiGAN model; the AAE model learns the latent feature distribution of normal samples through an encoder-generator structure, and the BiGAN model obtains the ability to generate and discriminate new samples through adversarial training between the generator and the encoder. The core detection and alarm module uses a two-level detection framework for traffic intrusion detection. The first level of detection uses the AAE model or BiGAN model to identify possible abnormal traffic; the second level uses the KNN classifier to determine the attack type. The results review and feedback module caches traffic features for each judgment result and reviews the judgment results of all traffic during that period at regular intervals. Based on the actual review results, it calculates the correct affirmative rate (TPR), correct negative rate (TNR), false affirmative rate (FPR), false negative rate (FNR), and unknown category ratio (UNR) of the traffic. It also adjusts the model used for the first-level detection based on the unknown category ratio (UNR) and adjusts the training samples of the generative deep learning model based on the correct affirmative rate (TPR), correct negative rate (TNR), false affirmative rate (FPR), and false negative rate (FNR).

Images