Lightweight intrusion detection method for Internet of Things based on traffic feature reconstruction and mapping

Abstract

The present invention discloses a lightweight intrusion detection method for the Internet of Things based on flow feature reconstruction and mapping. The present invention relates to the field of Internet of Things detection technology, and includes the following steps: collecting network flow data from Internet of Things devices, including normal flow data and potential abnormal flow data. The lightweight intrusion detection method for the Internet of Things based on flow feature reconstruction and mapping can more accurately identify and distinguish normal and abnormal flows through flow feature reconstruction and mapping, extract key information that best represents flow characteristics through feature reconstruction, and convert the extracted key information into a form suitable for machine learning model processing through mapping technology, which can not only reduce the interference of noise and irrelevant features, but also improve the sensitivity of the model to abnormal behavior. In addition, through incremental learning and online learning algorithms, it can continuously adapt to new attack modes and behavior changes, thereby continuously improving the accuracy of detection.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things detection technology, and in particular to an Internet of Things lightweight intrusion detection method based on flow feature reconstruction and mapping. Background Art

[0002] The rapid development of Internet of Things technology has enabled more and more devices to be interconnected, forming various application scenarios such as smart homes, smart transportation, and industrial automation. However, the accompanying network security threats are also becoming increasingly serious, especially the intrusion detection system is facing huge challenges. Since Internet of Things devices are usually resource-constrained, but have high requirements for the response speed of security incidents, the traffic feature reconstruction technology can be used to extract and reconstruct the features of the traffic generated by Internet of Things devices to expand the feature dimension and enhance the differences between features. In conjunction with feature mapping technology, the reconstructed features can be mapped to a low-dimensional space, which facilitates the analysis of related features, so as to quickly identify potential abnormal behaviors and provide effective security protection for the Internet of Things environment.

[0003] In the existing technology, since the detection process mainly relies on known attack patterns and feature libraries, it has poor adaptability to unknown attacks. In addition, due to the complexity of the IoT environment and the diversity of attack methods, false alarms and missed alarms are prone to occur during intrusion detection. Therefore, how to discover and adapt to new attack patterns and abnormal behaviors to reduce the false alarm rate and missed alarm rate during intrusion detection is the problem we need to solve. To this end, a lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping is proposed. Summary of the invention

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: a lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping, comprising the following steps:

[0005] Step 1: Collect network traffic data from IoT devices, including normal traffic data and potential abnormal traffic data, to ensure the diversity and comprehensiveness of the data, and perform pre-processing operations such as denoising, formatting, and timestamp alignment on the network traffic data to ensure data quality, and obtain a network traffic sequence table;

[0006] Step 2: Extract anomaly identification features from the preprocessed network traffic data. The anomaly identification features include the length, frequency, protocol type, source / destination IP address, and port number of the network packet, and reconstruct the extracted anomaly identification features using feature selection algorithms and deep learning technology;

[0007] Step 3: Use clustering algorithm to map the reconstructed features to low-dimensional space to reduce computational complexity and improve detection speed, and obtain feature data set. Based on the mapped feature data set, the intrusion detection model is constructed in combination with machine learning algorithm to identify abnormal behavior.

[0008] Step 4: Use an incremental learning algorithm to enable the intrusion detection model to learn and adapt to new attack patterns and abnormal behaviors, so as to update the feature library and intrusion detection model and identify attack patterns and abnormal behaviors;

[0009] Step 5: Combine the constructed intrusion detection model with the severity of abnormal behavior, calculate the abnormal warning coefficient, evaluate the severity of the detected abnormal behavior, set different abnormal levels, match the abnormal behavior and preset the corresponding level of warning notification;

[0010] Step 6: Deploy the intrusion detection model to the IoT environment for real-time detection, collect new traffic data, and detect and warn of abnormal behaviors in the IoT environment.

[0011] Preferably, in step 1, the process of obtaining the network traffic sequence table includes:

[0012] Identify IoT devices that serve as data sources, including smart cameras, sensors, and smart gateways, and configure network interfaces on IoT devices. Use network packet capture tools (Wireshark) and the built-in traffic monitoring function of IoT devices to capture network traffic data. Network traffic data includes normal traffic data and potential abnormal traffic data in different time periods, covering all possible network traffic scenarios as much as possible to ensure the comprehensiveness of the data.

[0013] Perform preprocessing operations on the captured network traffic data, including denoising, formatting, and timestamp alignment steps;

[0014] Extract key features from the preprocessed data, including source IP, destination IP, port number, timestamp, and packet size features, and store the preprocessed data in a data warehouse for subsequent analysis and query;

[0015] Perform time series analysis on the stored network traffic data to identify trends, periodicities, and anomalies in the data, and build a network traffic sequence table based on the time series analysis results. The network traffic sequence table contains timestamps, traffic size, number of data packets, and anomaly markers.

[0016] Preferably, in step 2, the process of reconstructing the abnormal identification feature includes:

[0017] Extract abnormal identification features based on the network traffic sequence table, including the length, frequency, protocol type, source / destination IP address, and port number of the network packet. For each data packet, calculate the specific value of its abnormal identification feature and normalize the feature value.

[0018] The feature selection algorithm of the filtering method is used to calculate the correlation between features, and the features most relevant to the target variable are selected based on statistical tests, and the contribution of each feature to anomaly detection is evaluated to reduce data dimensions, improve model performance and reduce overfitting;

[0019] Based on deep learning technology, an automatic encoder is used to reduce the dimension and reconstruct the features to obtain the hidden layer feature values, and a convolutional neural network is used to extract features from the time series data to calculate the reconstructed feature values.

[0020] Preferably, the calculation expression of the reconstructed eigenvalue is:

[0021]

[0022] Among them, F(x) is the reconstructed eigenvalue, x i is the original eigenvalue, w i is the feature weight, determined by the feature selection algorithm, n is the number of features, y j is the hidden layer feature value obtained by the autoencoder, m is the number of hidden layer feature values ​​obtained by the autoencoder, λ is the parameter affecting the adjustment exponential function, and j is the index used to traverse all hidden layer feature values ​​y in the summation process j .

[0023] Preferably, in step 3, the process of identifying abnormal behavior includes:

[0024] After feature reconstruction, the hierarchical clustering algorithm is used to perform cluster analysis on the reconstructed features, and the features are divided into different categories according to the similarity between the features;

[0025] Preprocess the reconstructed features, including normalization and noise removal, cluster the features using the selected clustering algorithm, map the features to a low-dimensional space, use the center point of each cluster as the representative point in the low-dimensional space, implement dimensionality reduction, and use the clustering results as new features to represent the category to which each data point belongs, thus obtaining a feature data set;

[0026] Based on the mapped low-dimensional feature data set, prepare training sets and test sets, where the training set is used to train the intrusion detection model, and the test set is used to evaluate the performance of the model. The intrusion detection model is trained using the training set data combined with the support vector machine algorithm to learn the patterns between normal and abnormal behaviors, thereby identifying new abnormal behaviors. The trained intrusion detection model is evaluated using the test set data, and the model is adjusted and optimized based on the evaluation results to improve its performance.

[0027] By using the associated data of the constructed intrusion detection model and the feature data set, based on the abnormal probability output by the intrusion detection model, a comprehensive analysis is performed to obtain the abnormal assessment index, identify potential abnormal behaviors, and evaluate the risk trend of the detected abnormal behaviors.

[0028] Preferably, the calculation expression of the abnormal evaluation index is:

[0029]

[0030] Among them, AEI is the abnormal assessment index, p i is the abnormal probability of the i-th data point, which is the output of the intrusion detection model, α i is the weight of the ith feature, reflecting the contribution of the feature to the abnormal assessment, b is the baseline value, which is the normal behavior probability threshold determined based on historical data, n is the number of features, max(p) and min(p) represent the probability of all p i , H is the set of high-risk abnormal behaviors, and ε is the adjustment index.

[0031] Preferably, in step 4, the process of updating the feature library and the intrusion detection model includes:

[0032] An online learning algorithm of the incremental learning algorithm is selected, the intrusion detection model is initialized using the training set data of the initial feature data set, and a network traffic real-time monitoring system is integrated on the intrusion detection model to receive real-time data streams, wherein the real-time data streams are derived from real-time data of network traffic and logs, and include new attack patterns and abnormal behaviors;

[0033] Perform preprocessing operations such as cleaning and normalization on real-time data streams to ensure data quality and consistency, extract relevant features for abnormal behavior detection from the preprocessed data, add the features associated with abnormal behavior detection in the new data to the training set, and use the online learning algorithm to train the intrusion detection model;

[0034] According to the results of incremental learning, the feature set in the feature library is updated. The new features represent new attack patterns and abnormal behaviors. Adding them to the feature library helps improve the detection capability of the model. The updated intrusion detection model is evaluated using test set data to verify the performance of the intrusion detection model. The updated intrusion detection model is deployed in the actual environment for use to identify attack patterns and abnormal behaviors.

[0035] Preferably, in step 5, the process of obtaining the abnormal warning coefficient includes:

[0036] Analyze the extent of damage caused by abnormal behavior based on the characteristic strength of abnormal behavior and the severity of historical similar cases, and define the severity indicators of abnormal behavior, including the scope of impact, duration and attack type;

[0037] Combine the severity index of abnormal behavior with the output of the intrusion detection model to calculate the abnormal warning coefficient and evaluate the severity and urgency of the detected abnormal behavior;

[0038] According to the range of the abnormal warning coefficient output results, different abnormal levels are set, namely low abnormal level, medium abnormal level and high abnormal level, and the warning threshold of the abnormal warning coefficient is set for each abnormal level. Abnormal behaviors exceeding the warning threshold are assigned to the corresponding abnormal level;

[0039] According to the calculated abnormal warning coefficient, each abnormal behavior is matched with the corresponding abnormal level, and a warning notification strategy is designed for each abnormal level, including notification method (email, SMS, application push), notification content, and urgency. When abnormal behavior is detected, a warning notification is automatically sent according to its abnormal level.

[0040] Preferably, the calculation expression of the abnormal warning coefficient is:

[0041]

[0042] Among them, AAC is the abnormal warning coefficient, w u is the weight coefficient, IMP u is the impact range indicator, which can be the number of affected users, devices or network areas, comprehensively reflecting the impact range of abnormal behavior. N is the number of impact range indicators. max(IMP) is the maximum value of all impact range indicators, which is used for normalization. T is the duration of abnormal behavior. is the attenuation factor of the duration, which is used to adjust the impact of the duration on the warning coefficient. AT is the severity score of the attack type, which is predefined based on historical data and knowledge base. θ is the baseline score of the attack type, which is used to judge the severity of the attack type. is the Sigmoid function, which is used to convert the attack type score into a value between 0 and 1, indicating the severity probability of the attack. r is the number of entities affected, such as the number of users or devices affected, y r Score the importance of the entity based on the role of the entity and the sensitivity of the data. M is the number of entity categories affected. k is the severity score of the kth entity affected, ω is the total number of affected entities, z avg It is the average of the severity scores of all entities affected. The value range of AAC is (0,1). As the severity and urgency of abnormal behavior increase, the value of AAC tends to 1.

[0043] The plurality of abnormal levels correspond to the plurality of warning thresholds, the warning thresholds include an upper threshold and a lower threshold, and the correspondence between the plurality of abnormal levels and the plurality of warning thresholds is:

[0044] Low abnormal level 0 <AAC≤AAC L ;

[0045] Medium Abnormal Level AAC L <AAC≤AAC M ;

[0046] High Abnormal Level AAC M <AAC<1;

[0047] Among them, MC is the abnormal warning coefficient, AAC L is the lower threshold corresponding to the medium abnormality level and the upper threshold corresponding to the low abnormality level, AAC M is the lower threshold corresponding to the high anomaly level and the upper threshold corresponding to the medium anomaly level, AAC L =0.3, AAC M =0.7.

[0048] Preferably, in step 6, the process of detecting and warning abnormal behavior in the IoT environment includes:

[0049] Deploy the selected intrusion detection model into the IoT environment, configure data collection points in IoT devices and networks to collect network traffic data, and use network monitoring tools to capture network traffic in real time. Pre-process the collected data, including denoising, formatting, and timestamp alignment.

[0050] The intrusion detection model performs real-time analysis on the pre-processed network traffic data, identifies potential abnormal behaviors, and classifies and detects abnormalities in the traffic data. Based on the model output results, it identifies the specific type of abnormal behavior.

[0051] Combine the severity index of abnormal behavior and the output results of the intrusion detection model to calculate the abnormal warning coefficient, evaluate the damage degree of abnormal behavior, and determine the abnormal level to which the abnormal behavior belongs, and then automatically send a warning notification based on the abnormal level;

[0052] Generate security reports regularly, including detected abnormal behaviors, warning events, and model performance, and regularly audit the response process of intrusion detection models to adapt the IoT environment to new threats.

[0053] The present invention provides a lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping. It has the following beneficial effects:

[0054] 1. This lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping can more accurately identify and distinguish normal and abnormal traffic through traffic feature reconstruction and mapping. Through feature reconstruction, the key information that best represents the traffic characteristics is extracted, and the extracted key information is converted into a form suitable for machine learning model processing through mapping technology. It can not only reduce the interference of noise and irrelevant features, but also improve the sensitivity of the model to abnormal behavior. In addition, through incremental learning and online learning algorithms, it can continuously adapt to new attack patterns and behavior changes, thereby continuously improving the accuracy of detection.

[0055] 2. This lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping can more accurately extract key features from network traffic through traffic feature reconstruction, and map these features to the intrusion detection model, effectively reducing the redundancy and noise of the original data, so that the intrusion detection model can focus more on analyzing features that are critical to identifying abnormal behaviors. In addition, by analyzing network traffic in real time, potential attack behaviors can be quickly identified. Once an anomaly is detected, an early warning mechanism is immediately triggered to notify relevant personnel and take corresponding defensive measures. The rapid response capability helps to reduce the damage caused by the attack and protect the overall security of the Internet of Things system. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The method flow chart of the lightweight intrusion detection method of the Internet of Things based on traffic feature reconstruction and mapping of the present invention;

[0057] Figure 2 A flow chart for identifying abnormal behavior for the present invention;

[0058] Figure 3 This is a flow chart for obtaining the abnormal warning coefficient of the present invention. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0060] The first embodiment, as Figure 1 , Figure 2 As shown, the present invention provides a technical solution: a lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping, comprising the following steps:

[0061] Step 1: Collect network traffic data from IoT devices, including normal traffic data and potential abnormal traffic data, to ensure data diversity and comprehensiveness, and perform preprocessing operations such as denoising, formatting, and timestamp alignment on the network traffic data to ensure data quality, obtain a network traffic sequence table, identify IoT devices as data sources, including smart cameras, sensors, and smart gateways, and configure network interfaces on IoT devices, use network packet capture tools (Wireshark) and the built-in traffic monitoring function of IoT devices to capture network traffic data, where network traffic data includes normal traffic data and potential abnormal traffic data in different time periods, covering all possible network traffic scenarios as much as possible to ensure data comprehensiveness, and perform preprocessing operations on the captured network traffic data, including denoising, formatting, and timestamp alignment steps, where the denoising step is performed through Filter irrelevant data, delete data packets irrelevant to network traffic analysis, such as broadcast packets, multicast packets, etc., and use statistical methods to detect and remove outliers, apply filters to reduce high-frequency noise in the data, convert all collected traffic data into a unified data format, use the network time protocol to synchronize the time sources of all IoT devices, and align the timestamps of the collected data to ensure the consistency of the data in the time series, extract key features from the preprocessed data, including source IP, destination IP, port number, timestamp, and data packet size features, and store the preprocessed data in the data warehouse for subsequent analysis and query, perform time series analysis on the stored network traffic data, identify trends, periodicity, and anomalies in the data, and build a network traffic sequence table based on the time series analysis results. The network traffic sequence table contains timestamps, traffic sizes, number of data packets, and anomaly markers;

[0062] Step 2: Extract anomaly identification features from the preprocessed network traffic data. The anomaly identification features include the length, frequency, protocol type, source / destination IP address, and port number of the network packet. The extracted anomaly identification features are reconstructed using feature selection algorithms and deep learning techniques. Anomaly identification features are extracted based on the network traffic sequence table, including the length, frequency, protocol type, source / destination IP address, and port number of the network packet. For each data packet, the specific value of its anomaly identification feature is calculated, and the feature value is normalized to eliminate the dimensional influence between different features. Among them, the network packet length is the size of the data packet, reflecting the data transmission mode or potential attack behavior. The frequency is the arrival rate of the data packet in a specific time window, which is used to detect traffic surge anomalies. The protocol type represents The network protocols used (such as TCP, UDP, ICMP, etc.), some protocols are more vulnerable to attacks, the source / destination IP address is the source and destination address of the data packet, which is used to identify possible attack sources or targets, the port number is the port number in the TCP / UDP data packet, and a specific port is associated with a specific service or attack mode. The feature selection algorithm of the filtering method is used to calculate the correlation between features, and the features most relevant to the target variable are selected based on statistical tests. The contribution of each feature to anomaly detection is evaluated to reduce data dimensions, improve model performance and reduce overfitting. The autoencoder is used based on deep learning technology to reduce the dimension and reconstruct the features, and the hidden layer feature values ​​are obtained. The convolutional neural network is used to extract features from time series data, and then the reconstructed feature values ​​are calculated;

[0063] Furthermore, the calculation expression of the reconstructed eigenvalue is:

[0064]

[0065] Among them, F(x) is the reconstructed eigenvalue, x i is the original eigenvalue, w i is the feature weight, determined by the feature selection algorithm, n is the number of features, y j is the hidden layer feature value obtained by the autoencoder, m is the number of hidden layer feature values ​​obtained by the autoencoder, λ is the parameter affecting the adjustment exponential function, and j is the index used to traverse all hidden layer feature values ​​y in the summation process j , represents the Euclidean norm of the weighted eigenvalues, which increases as the eigenvalues ​​increase. Indicates that with y j The increase of reflects the compression and abstraction of features by the deep learning model;

[0066] Step 3: Use clustering algorithms to map the reconstructed features to low-dimensional space to reduce computational complexity and improve detection speed, and obtain feature data sets. Based on the mapped feature data sets, an intrusion detection model is constructed in combination with a machine learning algorithm to identify abnormal behaviors. After feature reconstruction, a hierarchical clustering algorithm is used to perform cluster analysis on the reconstructed features. The features are divided into different categories according to the similarity between the features. The reconstructed features are preprocessed, including normalization and noise removal. The features are clustered using the selected clustering algorithm and mapped to low-dimensional space. The center point of each cluster is used as a representative point in the low-dimensional space to achieve dimensionality reduction. The clustering results are used as new features to represent the category to which each data point belongs. A feature data set is obtained, and a training set and a test set are prepared based on the mapped low-dimensional feature data set, wherein the training set is used to train the intrusion detection model, and the test set is used to evaluate the performance of the model. The intrusion detection model is trained using the training set data in combination with a support vector machine algorithm to learn the pattern between normal behavior and abnormal behavior, thereby identifying new abnormal behavior, and the trained intrusion detection model is evaluated using the test set data. The model is adjusted and optimized according to the evaluation results to improve its performance, and the associated data of the constructed intrusion detection model and the feature data set are used to obtain an abnormal evaluation index based on the abnormal probability output by the intrusion detection model through comprehensive analysis, thereby identifying potential abnormal behaviors and evaluating the risk trend of the detected abnormal behaviors;

[0067] Furthermore, the calculation expression of the abnormal evaluation index is:

[0068]

[0069] Among them, AEI is the abnormal assessment index, p i is the abnormal probability of the i-th data point, which is the output of the intrusion detection model, α i is the weight of the ith feature, reflecting the contribution of the feature to the abnormal assessment, b is the baseline value, which is the normal behavior probability threshold determined based on historical data, n is the number of features, max(p) and min(p) represent the probability of all p i The maximum and minimum values ​​of H are the set of high-risk abnormal behaviors, ε is the adjustment index, which is used to control the impact of the exponential function on the overall evaluation, and exp is the natural exponential function, which is used to deal with abnormal probabilities higher than the normal threshold. When p i When it is close to b and the difference is small, the AEI value will be low, indicating normal behavior. i When the difference from b increases, the denominator will increase the value of AEI, indicating a potential abnormality. i When a certain high-risk threshold is exceeded, the exponential function portion decreases rapidly, indicating an extremely high-risk anomaly;

[0070] Step 4: Adopt an incremental learning algorithm to enable the intrusion detection model to learn and adapt to new attack patterns and abnormal behaviors, so as to update the feature library and the intrusion detection model, identify attack patterns and abnormal behaviors, select an online learning algorithm of the incremental learning algorithm, use the training set data of the initial feature data set to initialize the intrusion detection model, and integrate the network traffic real-time monitoring system on the intrusion detection model to receive real-time data streams, wherein the real-time data streams are derived from real-time data of network traffic and logs, and contain new attack patterns and abnormal behaviors, perform preprocessing operations of cleaning and normalization on the real-time data streams to ensure the quality and consistency of the data, and extract the associated features of abnormal behavior detection from the preprocessed data, add the features associated with abnormal behavior detection in the new data to the training set, use the online learning algorithm to train the intrusion detection model, update the feature set in the feature library according to the results of incremental learning, the new features represent new attack patterns and abnormal behaviors, and adding them to the feature library helps to improve the detection capability of the model, and use the test set data to evaluate the updated intrusion detection model, verify the performance of the intrusion detection model, and deploy the updated intrusion detection model in the actual environment for use to identify attack patterns and abnormal behaviors;

[0071] Step 5: Combine the constructed intrusion detection model with the severity of abnormal behavior, calculate the abnormal warning coefficient, evaluate the severity of the detected abnormal behavior, set different abnormal levels, match the abnormal behavior and preset the corresponding level of warning notification;

[0072] Step 6: Deploy the intrusion detection model to the IoT environment for real-time detection, collect new traffic data, and detect and warn of abnormal behaviors in the IoT environment.

[0073] The second embodiment is based on the first embodiment. Figure 3 As shown, in step 5, the process of obtaining the abnormal warning coefficient includes:

[0074] Based on the characteristic strength of abnormal behavior and the severity of similar historical cases, the degree of damage caused by abnormal behavior is analyzed, and the severity index of abnormal behavior is defined, including the scope of impact, duration and attack type. Combined with the severity index of abnormal behavior and the output results of intrusion detection model, the abnormal warning coefficient is calculated to evaluate the severity and urgency of the detected abnormal behavior. According to the range of the output results of the abnormal warning coefficient, different abnormal levels are set, namely low abnormal level, medium abnormal level and high abnormal level, and the warning threshold of the abnormal warning coefficient is set for each abnormal level. Abnormal behavior exceeding the warning threshold is assigned to the corresponding abnormal level. According to the calculated abnormal warning coefficient, each abnormal behavior is matched with the corresponding abnormal level, and a warning notification strategy is designed for each abnormal level, including notification method (email, SMS, application push), notification content, and urgency. When abnormal behavior is detected, a warning notification is automatically sent according to its abnormal level. The calculation expression of the abnormal warning coefficient is:

[0075]

[0076] Among them, AAC is the abnormal warning coefficient, w u is the weight coefficient, IMP u is the impact range indicator, which can be the number of affected users, devices or network areas, comprehensively reflecting the impact range of abnormal behavior. N is the number of impact range indicators. max(IMP) is the maximum value of all impact range indicators, which is used for normalization. T is the duration of abnormal behavior. is the attenuation factor of the duration, which is used to adjust the impact of the duration on the warning coefficient. AT is the severity score of the attack type, which is predefined based on historical data and knowledge base. θ is the baseline score of the attack type, which is used to judge the severity of the attack type. is the Sigmoid function, which is used to convert the attack type score into a value between 0 and 1, indicating the severity probability of the attack. r is the number of entities affected, such as the number of users or devices affected, y r Score the importance of the entity based on the role of the entity and the sensitivity of the data. M is the number of entity categories affected. k is the severity score of the kth entity affected, ω is the total number of affected entities, z avg The average severity score of all entities affected. The value of IMP increases with x. r and r increases with the increase of z, reflecting the larger scope of influence and higher entity importance. kWhen the value distribution is wider (i.e., the variance increases), the value of IMP also increases, reflecting a more serious impact range. The value range of AAC is (0, 1). As the severity and urgency of abnormal behavior increase, the value of AAC tends to 1. When IMP u is smaller (i.e., the impact range is smaller), the value of AAC will be lower. When T is longer, due to the effect of the exponential decay function, the value of AAC will decrease. When AT is higher (i.e., the attack type is more serious), the value of the Sigmoid function tends to 1, thus increasing the value of AAC;

[0077] Multiple anomaly levels correspond to multiple warning thresholds. The warning thresholds include an upper threshold and a lower threshold. The correspondence between multiple anomaly levels and multiple warning thresholds is as follows:

[0078] Low anomaly level 0 < AAC ≤ AAC L ; indicating that the abnormal behavior is less serious and does not require an immediate response. Only record the abnormal behavior for further analysis;

[0079] Medium anomaly level AAC L < AAC ≤ AAC M ; indicating that the abnormal behavior has medium severity and requires attention and notification to members of the security team in charge for further investigation;

[0080] High anomaly level AAC M < AAC < 1; indicating serious abnormal behavior, triggering an automated response mechanism and notifying the security team for an emergency response;

[0081] Among them, MC is the anomaly warning coefficient, AAC L is the lower threshold corresponding to the medium anomaly level and the upper threshold corresponding to the low anomaly level, AAC M is the lower threshold corresponding to the high anomaly level and the upper threshold corresponding to the medium anomaly level, AAC L = 0.3, AAC M = 0.7;

[0082] In step six, the process of detecting and warning abnormal behavior in the Internet of Things environment includes:

[0083] The selected intrusion detection model is deployed in the IoT environment. Data collection points are configured in IoT devices and networks to collect network traffic data. Network monitoring tools are used to capture network traffic in real time. The collected data is preprocessed, including denoising, formatting, and timestamp alignment. The intrusion detection model analyzes the preprocessed network traffic data in real time to identify potential abnormal behaviors, and classifies and detects abnormalities in the traffic data. According to the output results of the model, the specific type of abnormal behavior is identified. Combined with the severity index of the abnormal behavior and the output results of the intrusion detection model, the abnormal warning coefficient is calculated, the degree of damage of the abnormal behavior is evaluated, and the abnormal level of the abnormal behavior is determined. Then, warning notifications are automatically sent according to the abnormal level, and security reports are generated regularly, including detected abnormal behaviors, warning events, and model performance. The response process of the intrusion detection model is audited regularly to adapt the IoT environment to new threats.

[0084] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.

Claims

1. A lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping, characterized in that: The following steps are involved: Step 1: Collect network traffic data from IoT devices, including normal traffic data and potential abnormal traffic data, and perform preprocessing operations on the network traffic data to obtain a network traffic sequence table; Step 2: Extract anomaly recognition features from the preprocessed network traffic data, and reconstruct the extracted anomaly recognition features using feature selection algorithm and deep learning technology; Step 3: Use clustering algorithm to map the reconstructed features to low-dimensional space to obtain feature data set. Based on the mapped feature data set, combine machine learning algorithm to build an intrusion detection model to identify abnormal behavior. Step 4: Use an incremental learning algorithm to enable the intrusion detection model to learn and adapt to new attack patterns and abnormal behaviors, so as to update the feature library and intrusion detection model and identify attack patterns and abnormal behaviors; Step 5: Combine the constructed intrusion detection model with the severity of abnormal behavior, calculate the abnormal warning coefficient, evaluate the severity of the detected abnormal behavior, set different abnormal levels, match the abnormal behavior and preset the corresponding level of warning notification; Step 6: Deploy the intrusion detection model to the IoT environment for real-time detection, collect new traffic data, and detect and warn of abnormal behaviors in the IoT environment.

2. According to claim 1, the lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping is characterized in that: In the step 1, the process of obtaining the network traffic sequence table includes: Identify IoT devices that serve as data sources, including smart cameras, sensors, and smart gateways, and configure network interfaces on IoT devices. Use network packet capture tools and built-in traffic monitoring functions of IoT devices to capture network traffic data, including normal traffic data and potential abnormal traffic data in different time periods. Perform preprocessing operations on the captured network traffic data, including denoising, formatting, and timestamp alignment steps; Extract key features from the preprocessed data, including source IP, destination IP, port number, timestamp, and packet size features, and store the preprocessed data in a data warehouse; Perform time series analysis on the stored network traffic data to identify trends, periodicities, and anomalies in the data, and build a network traffic sequence table based on the time series analysis results. The network traffic sequence table contains timestamps, traffic size, number of data packets, and anomaly markers.

3. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 2 is characterized in that: In the step 2, the process of reconstructing the abnormal identification feature includes: Extract abnormal identification features based on the network traffic sequence table, including the length, frequency, protocol type, source / destination IP address, and port number of the network packet. For each data packet, calculate the specific value of its abnormal identification feature and normalize the feature value. The feature selection algorithm of the filtering method is used to calculate the correlation between features, and the features most relevant to the target variable are selected based on statistical tests to evaluate the contribution of each feature to anomaly detection; Based on deep learning technology, an automatic encoder is used to reduce the dimension and reconstruct the features to obtain the hidden layer feature values, and a convolutional neural network is used to extract features from the time series data to calculate the reconstructed feature values.

4. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 3 is characterized in that: The calculation expression of the reconstructed eigenvalue is: Among them, F(x) is the reconstructed eigenvalue, x i is the original eigenvalue, w i is the feature weight, n is the number of features, y j is the hidden layer feature value obtained by the autoencoder, m is the number of hidden layer feature values ​​obtained by the autoencoder, λ is the parameter affecting the adjustment exponential function, and j is the index used to traverse all hidden layer feature values ​​y in the summation process j .

5. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 4 is characterized in that: In step 3, the process of identifying abnormal behavior includes: After feature reconstruction, the hierarchical clustering algorithm is used to perform cluster analysis on the reconstructed features, and the features are divided into different categories according to the similarity between the features; Preprocess the reconstructed features, including normalization and noise removal, cluster the features using the selected clustering algorithm, map the features to a low-dimensional space, use the center point of each cluster as the representative point in the low-dimensional space, implement dimensionality reduction, and use the clustering results as new features to represent the category to which each data point belongs, thus obtaining a feature data set; Based on the mapped low-dimensional feature data set, prepare training sets and test sets, use the training set data combined with the support vector machine algorithm to train the intrusion detection model, learn the patterns between normal and abnormal behaviors, and thus identify new abnormal behaviors. Use the test set data to evaluate the trained intrusion detection model, and adjust and optimize the model based on the evaluation results. By using the associated data of the constructed intrusion detection model and the feature data set, based on the abnormal probability output by the intrusion detection model, a comprehensive analysis is performed to obtain the abnormal assessment index, identify potential abnormal behaviors, and evaluate the risk trend of the detected abnormal behaviors.

6. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 5 is characterized in that: The calculation expression of the abnormal evaluation index is: Among them, AEI is the abnormal assessment index, p i is the abnormal probability of the i-th data point, α i is the weight of the ith feature, b is the baseline value, which is the normal behavior probability threshold determined based on historical data, n is the number of features, max(p) and min(p) represent all p i , H is the set of high-risk abnormal behaviors, and ε is the adjustment index.

7. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 6 is characterized in that: In step 4, the process of updating the feature library and the intrusion detection model includes: An online learning algorithm of the incremental learning algorithm is selected, the intrusion detection model is initialized using the training set data of the initial feature data set, and a network traffic real-time monitoring system is integrated on the intrusion detection model to receive real-time data streams, wherein the real-time data streams are derived from real-time data of network traffic and logs, and include new attack patterns and abnormal behaviors; Perform preprocessing operations such as cleaning and normalization on the real-time data stream, extract the associated features of abnormal behavior detection from the preprocessed data, add the features associated with abnormal behavior detection in the new data to the training set, and use the online learning algorithm to train the intrusion detection model; According to the results of incremental learning, the feature set in the feature library is updated. The new features represent new attack patterns and abnormal behaviors. The updated intrusion detection model is evaluated using the test set data and deployed in the actual environment to identify attack patterns and abnormal behaviors.

8. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 7 is characterized in that: In step 5, the process of obtaining the abnormal warning coefficient includes: Analyze the extent of damage caused by abnormal behavior based on the characteristic strength of abnormal behavior and the severity of historical similar cases, and define the severity indicators of abnormal behavior, including the scope of impact, duration and attack type; Combine the severity index of abnormal behavior with the output of the intrusion detection model to calculate the abnormal warning coefficient and evaluate the severity and urgency of the detected abnormal behavior; According to the range of the abnormal warning coefficient output results, different abnormal levels are set, namely low abnormal level, medium abnormal level and high abnormal level, and the warning threshold of the abnormal warning coefficient is set for each abnormal level. Abnormal behaviors exceeding the warning threshold are assigned to the corresponding abnormal level; According to the calculated abnormal warning coefficient, each abnormal behavior is matched with the corresponding abnormal level, and a warning notification strategy is designed for each abnormal level, including notification method, notification content, and urgency. When abnormal behavior is detected, a warning notification is automatically sent according to its abnormal level.

9. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 8 is characterized in that: The calculation expression of the abnormal warning coefficient is: Among them, AAC is the abnormal warning coefficient, w u is the weight coefficient, IMP u is the impact range indicator, which comprehensively reflects the impact range of the abnormal behavior, N is the number of impact range indicators, max(IMP) is the maximum value of all impact range indicators, T is the duration of the abnormal behavior, is the decay factor of the duration, AT is the severity score of the attack type, which is predefined based on historical data and knowledge base, and θ is the attack type benchmark score, which is used to judge the severity of the attack type. is the Sigmoid function, which is used to convert the attack type score into a value between 0 and 1, indicating the severity probability of the attack. r is the number of entities affected, y r Score the importance of the entity based on the role of the entity and the sensitivity of the data. M is the number of entity categories affected. k is the severity score of the kth entity affected, ω is the total number of affected entities, z avg It is the average of the severity scores of all entities affected. The value range of AAC is (0,1). As the severity and urgency of abnormal behavior increase, the value of AAC tends to 1. The plurality of abnormal levels correspond to the plurality of warning thresholds, the warning thresholds include an upper threshold and a lower threshold, and the corresponding relationship between the plurality of abnormal levels and the plurality of warning thresholds is: Low abnormal level 0 <AAC≤AAC L ; Medium Abnormal Level AAC L <AAC≤AAC M ; High Abnormal Level AAC M <AAC<1; Among them, MC is the abnormal warning coefficient, AAC L is the lower threshold corresponding to the medium abnormality level and the upper threshold corresponding to the low abnormality level, AAC M It is the lower threshold corresponding to the high abnormality level and the upper threshold corresponding to the medium abnormality level.

10. The lightweight intrusion detection method for the Internet of Things based on traffic feature reconstruction and mapping according to claim 9 is characterized in that: In step 6, the process of detecting and warning abnormal behavior in the IoT environment includes: Deploy the selected intrusion detection model into the IoT environment, configure data collection points in IoT devices and networks to collect network traffic data, and use network monitoring tools to capture network traffic in real time, and pre-process the collected data, including denoising, formatting, and timestamp alignment; The intrusion detection model performs real-time analysis on the pre-processed network traffic data, identifies potential abnormal behaviors, and classifies and detects abnormalities in the traffic data. Based on the model output results, it identifies the specific type of abnormal behavior. Combine the severity index of abnormal behavior and the output of the intrusion detection model to calculate the abnormal warning coefficient, evaluate the damage degree of abnormal behavior, and determine the abnormal level to which the abnormal behavior belongs, and then automatically send a warning notification based on the abnormal level; Generate security reports regularly, including detected abnormal behaviors, warning events, and model performance, and regularly audit the response process of intrusion detection models to adapt the IoT environment to new threats.

Images