A lightweight internet of things intrusion detection method

Abstract

The present application relates to the technical field of network security, and in particular to a lightweight Internet of Things intrusion detection method, which detects through a lightweight Internet of Things intrusion detection model after training, and the model adopts a whole structure of bifurcated data preprocessing, low-rank feature projection word segmentation, deep separable convolution and lightweight network architecture cascade fusion, entropy regularization lightweight self-attention and label-free global pooling classification. Then, the model parameter quantity is greatly reduced through low-rank constraint and operator decomposition, combined with offline SMOTE oversampling, dynamic weighted cross-entropy and spectral boundary constraint optimization, and the detection accuracy of hidden low-frequency attacks is improved. The model parameter quantity constructed by the method is only 15290, the volume is reduced by 60.2% compared with the standard Transformer, and it is suitable for resource-constrained Internet of Things edge security protection scenarios.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and specifically to a lightweight Internet of Things (IoT) intrusion detection method. Background Technology

[0002] With the widespread application of Internet of Things (IoT) technology, the deep interconnection of massive numbers of devices has significantly expanded the attack surface of systems, leading to a proliferation of various network attack methods. Network Intrusion Detection Systems (NIDS), as a key line of defense against network threats, face significant technical bottlenecks when deployed in IoT environments. Currently, IoT network traffic is inherently characterized by high dimensionality, large volume, and severe class imbalance (abundant normal traffic and very little malicious traffic). Existing deep learning detection models struggle to simultaneously meet detection accuracy and resource constraints: while mainstream Convolutional Neural Networks (CNNs) are effective in extracting local spatial features, their receptive fields are limited, making it difficult to capture long-term network temporal dependencies; Recurrent Neural Networks (RNNs / LSTMs) are computationally expensive, difficult to parallelize, and prone to gradient vanishing; the standard Transformer architecture possesses powerful global semantic modeling capabilities, but the computational complexity of its self-attention mechanism increases quadratically with the sequence length, and the number of parameters is enormous, making it impossible to run in real-time on IoT edge devices with extremely limited computing power and memory. Summary of the Invention

[0003] The purpose of this invention is to provide a lightweight IoT intrusion detection method, which aims to solve the problems of traditional intrusion detection models having a large number of parameters and high inference latency in IoT environments, making them unsuitable for deployment on resource-constrained IoT edge devices; at the same time, it addresses the problem that high-dimensional traffic and severe class imbalance lead to low model recognition rates for rare and covert attacks.

[0004] To achieve the above objectives, the present invention provides a lightweight Internet of Things (IoT) intrusion detection method, comprising the following steps:

[0005] Step 1: Data preprocessing and feature reconstruction;

[0006] Step 2: Cost-sensitive training and model optimization strategy settings;

[0007] Step 3: Lightweight feature projection and word segmentation;

[0008] Step 4: Depthwise separable convolution and residual fusion;

[0009] Step 5: Global modeling of lightweight network architecture;

[0010] Step 6: Unlabeled global feature pooling and classification output.

[0011] Optionally, in step 1, bifurcation preprocessing is performed on the original IoT traffic, and label encoding, missing value sentinel filling and one-hot encoding are performed on discrete category features to retain sparse semantics; Z-score normalization is performed independently on continuous numerical features to eliminate magnitude differences and retain abnormal contrast; then the two types of features are fused and horizontally stitched to form a two-dimensional matrix to complete data preprocessing and feature reconstruction.

[0012] Optionally, the training strategy set in step 2 may specifically be to balance the class distribution of the training set through offline oversampling using SMOTE; to improve the penalty for minority class attacks by using dynamic weighted cross-entropy loss; and to stabilize the training process and enhance the model's generalization ability by using cosine annealing Adam optimizer and spectral boundary constraints.

[0013] Optionally, the execution process of step 3 includes the following steps:

[0014] Step 3.1: Global context marker injection;

[0015] Step 3.2: Projection of low-rank subspace.

[0016] Optionally, during the execution of step 4, local spatiotemporal features are extracted through depthwise separable convolution, channel information is fused through pointwise convolution, and local features are fused with the original sequence using residual paths.

[0017] Optionally, step 5, the process of lightweight network architecture global modeling, includes the following steps:

[0018] Step 5.1: Pre-activation layer normalization and subspace multi-head self-attention;

[0019] Step 5.2: Entropy regularization constraint and attention distribution calculation;

[0020] Step 5.3: Parameterize the multilayer perceptron and connect it to the residuals.

[0021] Optionally, in step 6, label-free global average pooling is used to perform zero-parameter feature aggregation, and then a lightweight linear layer is used to complete the classification output.

[0022] This invention provides a lightweight IoT intrusion detection method. It employs a lightweight IoT intrusion detection model trained before detection. The model utilizes a bifurcated data preprocessing, low-rank feature projection word segmentation, cascaded fusion of depthwise separable convolutions and a lightweight network architecture, entropy regularization-based lightweight self-attention, and label-free global pooling classification. Furthermore, low-rank constraints and operator decomposition significantly reduce the number of model parameters. Combined with offline SMOTE oversampling, dynamic weighted cross-entropy, and spectral boundary constraint optimization, the detection accuracy against covert low-frequency attacks is improved. The model constructed using this method has only 15,290 parameters, reducing its size by 60.2% compared to the standard Transformer, making it suitable for resource-constrained IoT edge security scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the principle architecture of a lightweight IoT intrusion detection method according to the present invention. Detailed Implementation

[0025] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0026] This invention provides a lightweight Internet of Things (IoT) intrusion detection method, comprising the following steps:

[0027] Step 1: Data preprocessing and feature reconstruction;

[0028] Step 2: Cost-sensitive training and model optimization strategy settings;

[0029] Step 3: Lightweight feature projection and word segmentation;

[0030] Step 4: Depthwise separable convolution and residual fusion;

[0031] Step 5: Global modeling of lightweight network architecture;

[0032] Step 6: Unlabeled global feature pooling and classification output.

[0033] Specifically, the method architecture of this invention is as follows: Figure 1As shown, it specifically includes three stages:

[0034] (1) Data preprocessing stage: A bifurcated preprocessing pipeline is adopted to strictly distinguish between processing discrete and numerical attributes. For discrete features, numerical label encoding is performed and specific sentinel values ​​are filled in. Then, they are converted into mutually exclusive multidimensional binary vectors through one-hot encoding without normalization to preserve sparse semantics. For continuous numerical features, Z-score normalization is applied independently to eliminate differences in magnitude while preserving the feature contrast of anomalies to the greatest extent. The data preprocessing stage corresponds to step 1 of the method.

[0035] (2) Feature Processing Stage: The preprocessed features are first passed through a lightweight linear projection layer, which uses a low-rank transformation mechanism with independent weights to map the high-dimensional input into a low-dimensional token sequence, and a constant scalar is pre-loaded as a global context token. Then, the cascaded feature fusion module is entered, which uses depthwise separable convolution to decouple spatial and channel feature extraction, extracting local spatiotemporal patterns with extremely low parameter counts, and injecting these local features into the backbone token sequence through residual paths. Finally, the lightweight network architecture encoder runs a multi-head self-attention mechanism in the compressed subspace to capture global dependencies, and combines an entropy regularization strategy to smooth the attention distribution, thereby suppressing traffic noise and stabilizing gradients. The feature processing stage corresponds to steps 3 to 5 of the method.

[0036] (3) Classification Output Stage: A token-free classification header architecture is adopted, and zero-parameter global average pooling (GAP) is used to aggregate the entire sequence dimension to generate a unified one-dimensional representation. Then, a lightweight linear layer is used to map to a specific number of attack categories, and the predicted probability distribution is output. The classification output stage is step 6 of this method.

[0037] Furthermore, step 2 involves setting up cost-sensitive training and model optimization strategies. During training, the class weights are dynamically adjusted using a weighted classification cross-entropy loss function and SMOTE synthetic oversampling of the training set to ensure the model remains highly sensitive to covert and low-frequency attacks.

[0038] The following description, in conjunction with specific embodiments and execution processes, further illustrates the process of processing a single IoT network traffic data point, from raw data input to final classification output.

[0039] Step 1: Data Preprocessing and Feature Reconstruction

[0040] When the system receives a piece of raw network traffic data (containing text-type discrete fields, continuous numerical fields, and possible missing values), in order to prevent statistical distortion and preserve feature semantics, the system adopts a bifurcated preprocessing pipeline to strictly distinguish between processing discrete and numerical attributes.

[0041] 1. Discrete Category Feature Processing: For discrete attributes such as protocol type and service type, numerical label encoding is first performed. If missing values ​​exist, specific classification sentinel values ​​(such as the constant 999) are filled in, and then one-hot encoding is used to transform them into mutually exclusive multi-dimensional binary vectors. This process also transforms missing states into an independent dimension, completely discarding the significance of their numerical magnitude, and does not perform normalization operations to preserve sparse semantics.

[0042] 2. Continuous Numerical Feature Processing: For continuous features such as packet length and duration, the system independently applies Z-score normalization. The calculation formula is as follows:

[0043]

[0044] in, These are the original continuous eigenvalues. The mean of this feature for the training set, The standard deviation is denoted as . This operation standardizes the scale while preserving the numerical contrast caused by outliers. After the above bifurcation process, the results are merged to form a two-dimensional matrix for batch processing. ,in This represents the total dimension of the original numerical features.

[0045] Step 2: Cost-Sensitive Training and Model Optimization

[0046] To address the problem of model prediction bias caused by the extreme scarcity of minority attack categories in IoT data, the following optimizations were implemented during the model training phase:

[0047] 1. Offline Synthesis Oversampling: The SMOTE algorithm is strictly executed only on the partitioned training set to synthesize and increase the samples of rare classes to the baseline, ensuring that no data leakage occurs to the validation or test set.

[0048] 2. Dynamic weighted cross-entropy loss calculation: Recalculate dynamic class weights based on the resampled distribution. The predicted probability is obtained through forward propagation during training. Then, calculate the weighted classification cross-entropy (WCE) loss:

[0049]

[0050] in, The true unique hot label value for the c-th category. This sets the total number of attack categories (including normal classes) defined in the dataset, forcing the model to increase the penalty for incorrect predictions of covert, low-frequency attacks.

[0051] 3. Steady-state optimization based on spectral bound constraints: An Adam optimizer with cosine annealing and learning rate decay is employed, and a separate Frobenius norm penalty is uniformly applied to all attention and fully connected layer weights. (The Frobenius norm is the square root of the sum of the squares of all elements in a matrix, used to constrain the spectral norm of the weight matrix, thereby limiting the Lipschitz constant of the network and enhancing the model's generalization stability.) This step avoids the computationally expensive explicit orthogonalization operation while strictly constraining the network's Lipschitz constant, forcing the model to maintain high generalization ability and robustness with a very small number of parameters.

[0052] Step 3: Lightweight Feature Projection and Word Segmentation

[0053] To address the parameter explosion problem caused by directly inputting high-dimensional features into the Transformer, this architecture introduces an independent linear projection mechanism in the feature space.

[0054] 1. Global Context Labeling Injection: Concatenate a constant scalar to the beginning of the feature vector. As a global context token, it forms an enhancement vector. The total length of the sequence becomes The role of this marker is to act as a convergence hub in subsequent attention layers, promoting the fusion of global information among different features.

[0055] 2. Low-rank subspace projection: for each scalar element in the vector Utilizing its unique weight vector and bias vector Linearly map it to a dimension and set it as In the low-rank subspace, the formula for calculating the i-th embedding token is:

[0056]

[0057] in, By assigning independent projection weights to each feature index, this operation achieves significant dimensionality reduction while implicitly preserving the positional structure of heterogeneous network features, completely eliminating the need for explicit positional encoding matrices, and outputting a two-dimensional sequence matrix. .

[0058] Step 4: Depthwise Separable Convolution and Residual Fusion. To capture spatiotemporal patterns with local bursts, such as DoS attacks, in IoT traffic, the sequence matrix is ​​input into the depthwise separable convolution module. This module decouples the extraction processes of spatial features and channel features, greatly reducing computational overhead.

[0059] 1. Depthwise Convolution: Let the input tensor be... The system first performs depthwise convolutions independently across channels:

[0060]

[0061] in, The kernel is a deep convolutional layer. After extracting local temporal features, the output is processed by the ReLU activation function. .

[0062] 2. Pointwise convolution: Subsequently, cross-channel information is fused using pointwise convolution:

[0063]

[0064] in, This represents the weight matrix for pointwise convolution.

[0065] 3. Residual Fusion: To construct a cascaded feature fusion path, the system introduces element-level residual operations. The original labeled sequence X is convolved with the local spatiotemporal features extracted. Add directly:

[0066]

[0067] Since the spatial and channel dimensions of both are completely identical, this operation does not require dimension broadcasting and directly outputs the fused tensor. This is intended for use in the next stage, preserving the original features while mitigating gradient vanishing.

[0068] Step 5: Lightweight Network Architecture Global Modeling and Fusion of Local Feature Tensor Z. This is then fed into the lightweight network architecture encoder. The encoder employs a pre-activation layer normalization (Pre-LN) structure to stabilize training gradients and uses entropy regularization to constrain the attention distribution to suppress traffic noise. The specific execution process includes the following steps:

[0069] Step 5.1: Pre-activation layer normalization and subspace multi-head self-attention

[0070] The input tensor Z is first subjected to layer normalization to obtain the normalized features. To avoid the high computational burden of standard attention mechanisms, the attention mechanism is strictly restricted to operating in a low-dimensional subspace. Let the number of attention heads be... The dimensions of each head are .right The query matrix is ​​obtained by performing linear projection. Key matrix Sum matrix :

[0071] in The projection matrix is ​​learnable. Then, the scaled dot product attention score is calculated:

[0072]

[0073] scaling factor This is used to prevent the softmax gradient from disappearing due to excessively large dot product values.

[0074] Step 5.2: Entropy Regularization Constraints and Attention Distribution Calculation

[0075] To suppress noise interference in IoT traffic, the attention alignment process is modeled as an entropy maximization objective function, and the attention score is adjusted accordingly. By applying entropy regularization constraints, a smooth attention probability distribution is obtained:

[0076]

[0077] At the same time, an entropy regularization term is introduced. This encourages a more even distribution of attention, avoiding excessive focus on noise features. The element in the i-th row and j-th column of the attention score matrix A represents the original attention score of the i-th query position to the j-th key position. Similarly, the summation index k in the denominator traverses all positions in the sequence and is used to normalize all scores in the i-th row. Let be the element in the i-th row and j-th column of the attention probability distribution matrix P, representing the attention weight of the i-th position to the j-th position. This is the regularization coefficient, and its value range is usually [value range missing]. The larger this value, the more uniform the attention distribution and the stronger the noise suppression capability. In actual calculations, a closed-form solution is used, ultimately outputting the global semantic representation of the current header:

[0078]

[0079] in Let h be the probability distribution matrix of the h-th attention head. Let h be the value matrix of the h-th head. Multiplying the two matrices together yields the weighted semantic features of that head.

[0080] The output of the multi-head attention is obtained by concatenating the outputs of all heads and performing a linear transformation. , , ,in Here This indicates a concatenation operation along the feature dimension. The output projection matrix is ​​used to fuse multi-head information back into the original embedding dimension. Then, the attention output is compared with the original input. Perform residual join:

[0081]

[0082] Step 5.3: Parameterized Multilayer Perceptron and Residual Connection

[0083] Will After undergoing layer normalization again, the data is input into a compressed-expanded multilayer perceptron (MLP). The expansion factor is then set. (generally To balance expressive power and parameter cost. The inflation factor r represents the hidden layer dimension of the MLP relative to the input dimension. The magnification factor, i.e., the hidden layer dimension is The larger this value, the stronger the model's expressive power, but the number of parameters also increases accordingly. The calculation process of MLP is as follows:

[0084]

[0085] in For an extended matrix, For the corresponding bias; For compression matrix, For the corresponding bias, residual connections are introduced again to obtain the final output of the lightweight network architecture encoder:

[0086]

[0087] Repeat the above steps layer( By setting the encoder layer number, global dependency modeling can be completed.

[0088] Step Six: Label-free global feature pooling and classification output

[0089] 1. Global Feature Aggregation: After multiple lightweight network layers, to eliminate parameter redundancy caused by traditional classifier tokens, the system adopts a strictly unlabeled aggregation strategy. Let the final output sequence be... The system performs zero-parameter global average pooling (GAP) over the entire sequence dimension L:

[0090]

[0091] Output a unified aggregated one-dimensional representation .

[0092] 2. Classification Mapping: Finally, the final result is output through a lightweight linear classification layer:

[0093]

[0094] in, This is the classification weight matrix. The total number of attack categories defined in the dataset. Output. This refers to the predicted probability distribution of whether the network traffic belongs to each category of attack or normal traffic.

[0095] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0096] 1. Extremely high efficiency and low resource consumption: This invention reduces the model size by about 60.2% compared to the standard Transformer through low-rank constraints and operator decomposition at the architecture level, with only 15,290 parameters. This enables the system to maintain millisecond-level high-speed inference in IoT environments with limited CPU or GPU resources, fully meeting the real-time defense requirements of the edge.

[0097] 2. Enhanced detection capabilities: The design of cascaded convolution and lightweight network architecture achieves complementarity between local spatial awareness and global semantic modeling. This allows the model to maintain extreme lightweightness while exhibiting extremely high recognition accuracy against attacks with local burst characteristics such as DoS, as well as long-term covert cross-protocol attacks.

[0098] 3. Effective handling of class imbalance: By combining cost-sensitive training strategies of offline enhancement and dynamic weight compensation, the model significantly improves its ability to capture low-frequency minority class attack samples, and exhibits stronger robustness and lower false negative rate in highly imbalanced real-world IoT traffic scenarios.

[0099] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A lightweight Internet of Things (IoT) intrusion detection method, characterized in that, Includes the following steps: Step 1: Data preprocessing and feature reconstruction; Step 2: Cost-sensitive training and model optimization strategy settings; Step 3: Lightweight feature projection and word segmentation; Step 4: Depthwise separable convolution and residual fusion; Step 5: Global modeling of lightweight network architecture; Step 6: Unlabeled global feature pooling and classification output.

2. The lightweight IoT intrusion detection method as described in claim 1, characterized in that, In step 1, bifurcation preprocessing is performed on the original IoT traffic. Discrete category features are labeled, filled with missing values ​​by sentinels, and encoded one-hot to preserve sparse semantics. Continuous numerical features are independently normalized using Z-score to eliminate magnitude differences and preserve abnormal contrast. The two types of features are then fused and horizontally stitched together to form a two-dimensional matrix, completing data preprocessing and feature reconstruction.

3. The lightweight IoT intrusion detection method as described in claim 2, characterized in that, The training strategy set in step 2 specifically involves balancing the class distribution of the training set through offline oversampling using SMOTE; increasing the penalty for minority class attacks by using dynamic weighted cross-entropy loss; and stabilizing the training process and enhancing the model's generalization ability by using the cosine annealing Adam optimizer and spectral boundary constraints.

4. The lightweight IoT intrusion detection method as described in claim 3, characterized in that, The execution process of step 3 includes the following steps: Step 3.1: Global context marker injection; Step 3.2: Projection of low-rank subspace.

5. The lightweight IoT intrusion detection method as described in claim 4, characterized in that, During the execution of step 4, local spatiotemporal features are extracted through depthwise separable convolution, channel information is fused through pointwise convolution, and local features are fused with the original sequence using residual paths.

6. The lightweight IoT intrusion detection method as described in claim 5, characterized in that, Step 5, the process of global modeling of lightweight network architecture, includes the following steps: Step 5.1: Pre-activation layer normalization and subspace multi-head self-attention; Step 5.2: Entropy regularization constraint and attention distribution calculation; Step 5.3: Parameterize the multilayer perceptron and connect it to the residuals.

7. The lightweight IoT intrusion detection method as described in claim 6, characterized in that, In step 6, label-free global average pooling is used to perform zero-parameter feature aggregation, and then a lightweight linear layer is used to complete the classification output.

Images