An internet of things device identification method and system based on semantic perception representation and adaptive semantic feedback

Abstract

The application discloses a kind of based on semantic perception representation and adaptive semantic feedback's internet of things equipment identification method and system.The method is according to physical semantic logic to the original traffic statistical characteristics is grouped and is reconstructed, utilizes nonlinear transformation and histogram equalization to enhance flow texture difference, feature is mapped as the fingerprint image with physical semantic association.In feature extraction stage, construct by cascaded multi-scale convolution module, dynamic gate residual module and mixed space-time feature module Composition's sequence captures local space-time correlation characteristics;Smooth context semantic component and gradient enhancement component are respectively extracted using the double-path architecture of adaptive decoupling residual module, after being weighted after channel level adaptive weight, feature mapping integration and residual fusion are executed, realize the decoupling modeling of smooth semantic feature and gradient response feature.Deep global semantic vector is concentrated using spatial pyramid pooling, and via linear mapping layer and feature linear modulation layer executes top-down channel level affine transformation, realizes shallow feature re-indexing.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) device identification technology, and in particular to an IoT device identification method and system based on semantic perception representation and adaptive semantic feedback. Background Technology

[0002] With the rapid development of IoT technology, massive numbers of heterogeneous devices are being deployed in various key areas, but the resulting cybersecurity risks are also becoming increasingly prominent. Due to the common problems of limited resources and lagging system updates, IoT devices are highly susceptible to becoming springboards for cyberattacks. Therefore, accurate identification of devices connected to the network is a primary prerequisite for effective security control. Device fingerprinting technology based on network communication behavior, as a passive authentication method, can effectively overcome the limitations of traditional physical identifiers (such as IP / MAC addresses) that are easily forged and tampered with, and has become a core technology in this field.

[0003] However, traditional device fingerprinting methods mostly rely on single-dimensional, shallow traffic features. When faced with jitter, packet loss, and increasingly prevalent encryption protocols in real-world network environments, these single features often exhibit severe instability and limitations in representation. To overcome this bottleneck, current technological trends are gradually evolving towards multi-dimensional feature transformation and deep pattern mining. By mapping network traffic to richer representation spaces (such as multi-scale spatial or frequency domain transformations), the aim is to more effectively capture the inherent communication patterns at the device's underlying level, thereby enhancing the anti-interference capability and robustness of device fingerprints and providing a more reliable technical path for IoT security protection in complex network environments.

[0004] The Chinese patent authorization number is "CN116958586B", entitled "A Method and System for IoT Device Identification Based on Multi-Scale Attention Capsule Networks". This method enhances the discriminative representation of key traffic features by introducing a multi-scale feature extraction architecture and attention mechanism into the feedforward network. It also utilizes the vector activation and coupling mechanism unique to capsule networks to attempt to preserve the topological and positional associations of traffic fingerprints in the spatial dimension, thereby improving the model's ability to capture underlying communication behavior features. However, this method suffers from problems in IoT device identification, including insufficient deep feature extraction, weak anti-confusion ability, low minority class recognition rate, and poor generalization performance in long-tail scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an IoT device identification method based on semantic perception representation and adaptive semantic feedback. This method solves the problems of existing IoT device identification methods, such as limited single feature expression, severe homogenization of encryption protocols, insufficient deep feature extraction due to extreme long-tail data distribution, easy confusion of highly similar devices, and low accuracy of minority class identification.

[0006] To achieve the above objectives, the present invention specifically adopts the following technical solution: A method and system for identifying IoT devices based on semantic perception representation and adaptive semantic feedback, comprising the following steps: Step 1, Prepare the dataset: Obtain the network traffic dataset of IoT devices and divide the dataset into training set, validation set and test set according to a certain ratio; Step 2, Data Preprocessing: Obtain the original network traffic statistical features, group and reconstruct the features according to the preset physical semantic logic; use nonlinear transformation to balance outliers, and apply histogram equalization algorithm to enhance traffic texture differences, mapping the feature data into a pseudo-color traffic fingerprint image with physical semantic association. Step 3, Constructing the Network Model: The entire IoT device identification network based on semantic perception representation and adaptive semantic feedback consists of a feature extraction module, a global context perception module, a semantic feedback modulation module, and a multi-task joint output module. Specifically, the feature extraction module comprises a multi-scale convolution module, a dynamic gated residual module, a hybrid spatiotemporal feature module, and an adaptive decoupling residual module; the global context perception module comprises a global differential attention module and a global context extraction module; the semantic feedback modulation module comprises a linear mapping layer and a feature linear modulation layer; and the multi-task joint output module comprises a classification mapping head and a contrastive feature projection head. Step 4: Select appropriate loss function and evaluation metrics: Select a joint loss function consisting of labeled smooth cross-entropy loss function and supervised contrastive learning loss function; use accuracy, precision, recall and macro average F1 score as performance evaluation metrics for the model. Step 5, Train the network model: Train the IoT device identification model. Preprocess the dataset from Step 1 using the method in Step 2, and input the preprocessed data into the IoT device identification model built in Step 3 for training. Step 6, Save the model: Solidify the parameters of the model that performs best in terms of evaluation metrics to determine the final IoT device identification model.

[0007] Furthermore, in step 1, dividing the dataset into training, validation, and test sets according to a certain ratio specifically involves: adopting a stratified sampling strategy to divide the network traffic dataset according to a preset ratio to ensure that the sample distribution ratio of various IoT devices in the training, validation, and test sets remains consistent.

[0008] Furthermore, in step 2, the grouping, reconstruction, and arrangement of features according to the preset physical semantic logic specifically includes: sequentially arranging the extracted network traffic statistics features in an ordered manner according to the physical semantic order of time-related features, packet count features, byte stream features, packet length distribution features, arrival interval features, TCP flag features, and connection behavior features.

[0009] Furthermore, in step 3, the adaptive decoupling residual module in the feature extraction module works as follows: the adaptive decoupling residual module consists of a main semantic branch and a gradient enhancement branch. The main semantic branch extracts smooth contextual semantic components to capture stable semantic patterns; the gradient enhancement branch captures gradient change information through a multi-scale difference operator group containing asynchronous long misaligned difference operations, and generates gradient enhancement components through cross-channel feature fusion and adaptive modulation; then, the two components are weighted and mapped and the residuals are added to achieve branch decoupling modeling of smooth semantic features and gradient response features.

[0010] Furthermore, in step 3, the interaction process between the global context perception module and the semantic feedback modulation module is as follows: the global differential attention module constructs the global long-range dependency relationship of features through a multi-head self-attention mechanism, and the global context extraction module obtains the global semantic vector through spatial pyramid pooling; subsequently, the linear mapping layer in the semantic feedback modulation module generates dynamic scaling parameters and bias parameters according to the global semantic vector, and the feature linear modulation layer performs channel-level affine transformation on the shallow features output by the feature extraction module to achieve top-down feature recalibration.

[0011] Furthermore, in step 3, the specific working method of the multi-task joint output module is as follows: the classification mapping head outputs the classification probability distribution of device identity through a fully connected mapping; the comparison feature projection head maps the fused features to a high-dimensional hypersphere, and through comparison clustering, it brings the features of similar devices closer together and pushes away the features of dissimilar devices.

[0012] Furthermore, in step 4, the joint loss function is a weighted combination of labeled smooth cross-entropy loss and supervised contrastive learning loss; the former suppresses the model's overconfidence in the majority of devices through a soft label mechanism to alleviate long-tail bias; the latter promotes the formation of compact and well-defined clusters of similar features in the latent space through contrastive constraints; the two work together to perform multi-task joint optimization, and simultaneously complete device classification decisions and feature space arrangement.

[0013] Furthermore, in step 5, during the training process of the input model, the following steps are also included: using a random sampling strategy based on square root smoothing weights to perform weighted sampling on the training set data, and applying Mixup data obfuscation technology to generate virtual enhanced samples in the feature space, so as to adaptively adjust the discrimination boundary of unbalanced samples and alleviate the long-tail distribution bias in the scenario of extremely small samples.

[0014] Compared with existing technologies, this invention provides an IoT device identification method based on semantic perception representation and adaptive semantic feedback, which has the following beneficial effects: This invention provides an IoT device identification method based on semantic perception representation and adaptive semantic feedback. By constructing a closed-loop network architecture with multi-level feature perception and adaptive feedback, it achieves comprehensive capture and deep fusion of underlying physical traffic fingerprints and deep protocol context features. This overall architecture effectively breaks through the limitations of traditional single-feature representation, improving the model's device classification accuracy, anti-aliasing ability, and overall robustness in the face of unknown environments in complex network environments.

[0015] This invention proposes an adaptive decoupled residual module in the feature extraction module. By constructing a dual-path parallel processing architecture, it achieves decoupled modeling of smooth semantic features and gradient response features. Multi-scale differential operators integrated within the branches are used to extract gradient change information of traffic features at different scales, generating adaptively modulated gradient enhancement components. These components are then weighted and fused with the smooth contextual semantic components extracted from the main semantic branch, followed by feature mapping. This decoupling mechanism enables the model to capture the inherent smooth semantic patterns and significant gradient changes of devices in noisy environments with network jitter or packet loss, improving the stability of feature extraction.

[0016] This invention constructs a global semantic feedback mechanism, utilizing a global context-aware module to extract the global semantics of deep protocols, and generating dynamic affine parameters through a semantic feedback modulation module to perform channel-level recalibration on shallow features. This dynamic feedback architecture can suppress redundant similar noise from devices of the same manufacturer or type, and enhance the differences in micro-features between devices, thereby achieving effective device differentiation in encrypted traffic environments.

[0017] This invention employs a multi-task joint output and dynamic smoothing strategy. On one hand, by constructing a contrastive feature projection head in the multi-task joint output module, supervised contrastive constraints are applied based on a high-dimensional feature space, prompting the features of similar few-sample devices to form compact clusters. On the other hand, combined with a labeled smoothing cross-entropy loss mechanism, the model's overconfidence prediction of large-sample majority-class devices is suppressed. This joint optimization strategy alleviates the problem of limited minority-class feature representation and improves the model's generalization performance and the recognition accuracy of minority-class devices under imbalanced data. Attached Figure Description

[0018] Figure 1 A flowchart of the IoT device identification method provided by the present invention; Figure 2 A flowchart for data preprocessing provided by the present invention; Figure 3 This is a general framework diagram of the IoT device identification method provided by the present invention; Figure 4 A schematic diagram illustrating the specific components of the multi-scale convolution module provided by this invention; Figure 5 A schematic diagram showing the specific composition of the dynamic gating residual module provided by the present invention; Figure 6-7 A schematic diagram illustrating the specific composition of the hybrid spatiotemporal feature module provided by this invention; Figure 8-9 This is a schematic diagram illustrating the specific composition of the adaptive decoupling residual module provided by the present invention; Figure 10-11 This is a schematic diagram illustrating the specific composition of the global differential attention module provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: like Figure 1 The diagram shown is a flowchart illustrating the overall steps of an IoT device identification method and system based on semantic perception representation and adaptive semantic feedback, according to an embodiment of the present invention. The method specifically includes the following steps: Step 1, Prepare the dataset: Prepare two public network traffic datasets, the AaltoUniversityIoT dataset and the UNSWIOT dataset, and divide the datasets into training set, validation set and test set according to a certain ratio.

[0021] Step 2, Data Preprocessing: Statistical features are extracted from the original traffic packets and feature filtering is performed. Then, the features are grouped and reconstructed according to physical semantics to form a one-dimensional sequence. Finally, a pseudo-color traffic fingerprint image with physical semantic association is generated by constructing a traffic matrix and combining logarithmic compression, normalization, contrast-limited adaptive histogram equalization, and pseudo-color mapping and other feature image enhancement operations.

[0022] Step 3, Constructing the Network Model: The entire IoT device identification network based on semantic perception representation and adaptive semantic feedback consists of a feature extraction module, a global context perception module, a semantic feedback modulation module, and a multi-task joint output module.

[0023] The feature extraction module consists of a multi-scale convolution module, a dynamic gated residual module, a hybrid spatiotemporal feature module, and an adaptive decoupling residual module.

[0024] The adaptive decoupling residual module consists of a main semantic branch and a gradient enhancement branch. The main semantic branch is responsible for extracting smooth contextual semantic components from the traffic features to capture stable semantic patterns. The gradient enhancement branch contains a multi-scale difference operator group with different time-length parallel staggered difference operations to extract gradient change information of the traffic feature image under different receptive ranges, and obtains the gradient enhancement component through internal feature fusion and adaptive modulation. The decoupled gradient enhancement component and smooth contextual semantic component are respectively weighted by channel-level adaptive weights and then input into the fusion projection layer for feature mapping and integration in spatial and channel dimensions. Subsequently, residual addition and fusion are performed with the input features through a jumper wire.

[0025] The global context awareness module consists of a global differential attention module and a global context extraction module. The global differential attention module uses a multi-head self-attention mechanism to capture the association of protocol sequences across time steps and extract global semantics. The global context extraction module uses spatial pyramid pooling and a multilayer perceptron to reduce the dimensionality of features, perform feature compression and channel statistical encoding, and output a global semantic vector with fixed dimensions, providing high-level contextual guidance for subsequent top-down feature modulation.

[0026] The semantic feedback modulation module consists of a linear mapping layer and a feature linear modulation layer. The linear mapping layer contains two sets of parallel fully connected networks, used to dynamically generate a scaling parameter vector γ and a bias parameter vector β that match the number of shallow physical feature channels, taking the global semantic vector output by the global context awareness module as input. The feature linear modulation layer receives the scaling parameter vector γ and the bias parameter vector β and directly applies them to the shallow physical features output by the dynamic gated residual module and the hybrid spatiotemporal feature module, by mapping the feature maps of each channel of the shallow features to the corresponding scaling parameter γ. c Perform element-wise multiplication and add the corresponding bias parameter β c It performs channel-level affine transformations without additional learnable convolution parameters; this module thereby establishes a dynamic recalibration mechanism that guides local features with global semantics, which is used to enhance the expressive power of discriminative features between devices and reduce feature similarity caused by protocol homomorphism.

[0027] The multi-task joint output module consists of a classification mapping head and a contrastive feature projection head. The contrastive feature projection head is composed of fully connected layers and activation functions, and uses an L2 normalization layer to map high-dimensional features onto a low-dimensional unit hypersphere. Based on this, cosine similarity is calculated and supervised contrast constraints are applied to constrain the formation of compact and well-defined clusters of similar few-sample devices in the latent space. The classification mapping head is composed of linear fully connected layers and a Softmax layer, outputting the predicted probability distribution of device identity. Combined with a labeled smoothing cross-entropy loss mechanism, it forcibly suppresses the model's overconfidence prediction of the majority class of large-sample devices. This module jointly outputs and co-optimizes the probability distribution-based classification decision and the geometric distance-based clustering features, effectively alleviating the problem of limited minority class representation caused by long-tail distribution in real IoT scenarios.

[0028] Step 4: Selecting appropriate loss functions and evaluation metrics: During training, the loss function selected is a multi-task joint loss function that is a weighted combination of labeled smooth cross-entropy loss and supervised contrastive learning loss. The labeled smooth cross-entropy loss is used to suppress the model's overconfidence prediction of large-sample devices of the majority class, while the supervised contrastive learning loss is used to promote the formation of compact clusters of features of small-sample devices of the same class. At the same time, multiple quantitative evaluation metrics are selected to jointly evaluate the model's recognition performance, including macro-average F1 score, overall classification accuracy, precision, recall, and a confusion matrix for visually displaying device misclassification.

[0029] Step 5, Train the network model: Train the IoT device identification network model. Preprocess the AaltoUniversityIoT dataset and UNSWIOT dataset prepared in Step 1 using the method in Step 2, and input the semantically aware traffic matrix image generated after preprocessing into the IoT device identification model constructed in Step 3 for training and optimization.

[0030] Step 6, Save the model: Solidify the model parameters that perform best on the validation set according to the evaluation metrics, and determine the final IoT device identification model based on semantic perception representation and adaptive semantic feedback.

[0031] Example 2: like Figure 1 As shown, a flowchart of an IoT device identification method and system based on semantic perception representation and adaptive semantic feedback is presented in one embodiment of the present invention. The method specifically includes the following steps: Step 1: Prepare Datasets. Two publicly available real-world IoT traffic datasets are prepared: the Aalto University IoT dataset and the UNSW Sydney SmartHome IoT dataset. The Aalto University IoT dataset was collected in a real smart home network environment and includes the underlying communication traffic of 31 typical IoT devices, such as smart speakers, smart sockets, and network cameras, during network configuration and normal operation. The UNSW IoT dataset was collected over several days from a specific publicly monitored environment and covers rich communication traffic from 28 typical commercial IoT devices, such as smart light bulbs, motion sensors, and smart scales. Both datasets are in PCAP file format and contain a large amount of real network data packets. To verify the generalization ability and training effectiveness of the proposed network model under multi-source heterogeneous device data, these two traffic datasets are cleaned and then divided into training, validation, and test sets in a 7:1:2 ratio.

[0032] Step 2, Data Preprocessing: Extract physical communication behavior features from the original PCAP traffic packets and convert them into feature matrix images with two-dimensional visual texture through spatial arrangement and pixel mapping to adapt to the local receptive field mechanism of convolutional neural networks. The specific process is as follows: Figure 2 As shown: First, statistical features such as packet length and arrival time interval are extracted from the original traffic packets and then filtered. Next, the features are grouped and reconstructed according to physical semantics to form a one-dimensional sequence, followed by the construction of a two-dimensional traffic matrix. To enhance the visual texture representation of the features and eliminate the influence of extreme values, the matrix is ​​sequentially subjected to logarithmic compression, normalization, and contrast-limited adaptive histogram equalization. The mathematical expressions for logarithmic compression and normalization are as follows: , , in, Represents the first in the original two-dimensional flow matrix Line 1 The characteristic values ​​of the column, This represents the logarithmically compressed value. This represents the normalized value. and These represent the maximum and minimum values ​​in the feature matrix, respectively.

[0033] Finally, the channel dimensions are generated through pseudo-color mapping. The size is fixed as The semantically aware traffic matrix image is used as network input.

[0034] Step 3, construct the network model: such as Figure 3The diagram shown illustrates the overall framework of an IoT device identification method and system based on semantic perception representation and adaptive semantic feedback provided by this invention. Specifically, it includes a feature extraction module, a global context perception module, a semantic feedback modulation module, and a multi-task joint output module. In this overall framework diagram, solid lines with arrows represent the forward data flow of the high-dimensional feature matrix between sub-units, while dashed streamlines with arrows represent the top-down semantic feedback control path.

[0035] The feature extraction module consists of a cascaded multi-scale convolution module, a dynamically gated residual module, a hybrid spatiotemporal feature module, and an adaptive decoupling residual module. The specific components of the multi-scale convolution module are as follows: Figure 4 As shown, this module is used to extract local spatiotemporal physical textures across multiple receptive fields. It comprises three parallel feature extraction branches. The first branch consists of two cascaded 3×3 convolutional blocks to extract local features at the basic scale. The second branch consists of cascaded 5×5 and 3×3 convolutions to obtain spatial features with a large receptive field. The third branch uses alternating concatenation of 1×7 and 7×1 asymmetric convolutions to control the number of model parameters while expanding the receptive field. The output features from the three branches are concatenated and aggregated along the channel dimension, then input to a fusion convolutional layer, a batch normalization layer, and a G-type activation function for cross-channel feature fusion and nonlinear mapping. After spatial resolution downsampling via a max-pooling layer, multi-scale fused features are output. The mathematical expression for multi-scale feature fusion is as follows: , in, , , These represent the multi-scale features output from the first, second, and third branches, respectively. This represents a splicing and aggregation operation along the channel dimension. and These represent the weights and biases of a 1×1 fused convolutional layer, respectively. Represents the G-type activation function. This indicates a max-pooling downsampling operation. This represents the final output feature of the multi-scale convolution module.

[0036] The specific components of the dynamic gated residual module are as follows: Figure 5As shown, the system consists of cascaded first and second gated residual units, used to adaptively control the flow ratio of residual information while capturing deep features. In the first gated residual unit, the input features are split into the main branch and the residual branch. The main branch contains a convolutional block consisting of a 3×3 convolutional layer, a normalization layer, and an activation function, and a linear mapping layer consisting of a 3×3 convolutional layer and a normalization layer, used to extract deep local spatial features. The output features of the main branch are input to an attention module consisting of an adaptive average pooling layer and a multilayer perceptron, and are then channel-statistically encoded and mapped by a sigmoid function to generate a dynamic gated weight vector. Since this unit involves spatial resolution downsampling, the residual branch achieves spatial and channel dimension alignment via a 1×1 convolutional layer and a normalization layer. This unit employs a complementary gating fusion mechanism, combining the main branch features with dynamic weights. Element-wise multiplication simultaneously incorporates residual branch features and complementary weights. Element-wise multiplication is performed, and the two weighted features are fused through an addition node and output by a G-type activation function. A similar two-branch processing architecture is used in the second gated residual unit. The main branch extracts features and generates gate weights sequentially through the second convolutional block and the corresponding linear mapping layer; since the spatial and channel dimensions of the features are aligned at this point, the residual branch directly uses an identity mapping shorting line. The two features are fused using the same complementary weighting mechanism and nonlinearly mapped by a G-type activation function, outputting the final fused feature of this module. The mathematical expression of its complementary gated fusion mechanism is as follows: , in, Indicates main branch characteristics Indicates residual branch characteristics, This represents the dynamic gating weight vector generated by the attention module. Indicates complementary weights, This indicates an element-wise multiplication operation. Represents the G-type activation function. This indicates the feature fusion output of this unit.

[0037] The structure of the hybrid spatiotemporal feature module is as follows: Figure 6 As shown, this module consists of cascaded temporal convolutional blocks, an axial attention module, and a G-type activation function, used to mine local load patterns and global temporal dependencies in IoT traffic. The internal structure of the temporal convolutional block is as follows: Figure 7As shown, a parallel multi-branch architecture combining standard convolution and asymmetric convolution is adopted. This module includes a spatial awareness branch and an axial decomposition branch. The spatial awareness branch consists of cascaded 3×3 convolutional layers, normalization layers, and G-type activation functions to extract the width dimension information of the feature image; the axial decomposition branch extracts features sequentially through 3×1 convolutional blocks and 1×3 convolutional blocks, using the asymmetric convolutional structure to reduce the number of model parameters while stripping away spatiotemporal dimension features. The output features of the above two branches are concatenated and aggregated in the channel dimension, input to a 1×1 convolutional layer as a channel fusion layer to perform cross-channel information interaction, and then added and fused with the input features through residual jumpers to generate a temporal convolutional block feature map.

[0038] The axial attention module is cascaded after the temporal convolutional block and is used to perform multi-dimensional feature recalibration. Its internal structure is as follows: Figure 8 As shown, this module comprises parallel channel attention branches, height attention branches, and width attention branches. Input features are fed into these three branches in parallel, where attention weight distributions are independently calculated along the channel axis of the feature map, the height axis of the mapped time series, and the width axis of the mapped physical payload, respectively. The channel attention branch consists of an adaptive average pooling layer, a 1×1 convolutional layer, and a sigmoid activation function; the height and width attention branches generate corresponding spatial attention masks using operators based on grouped one-dimensional convolution. The three sets of axial attention weights are adaptively weighted and multiplied element-wise with the input features. The fused features are then passed through a fusion convolutional layer, a normalization layer, and a G-type activation function for feature condensation, and finally added to the input features of the axial attention module via a residual jumper. The mathematical expression is as follows: , in, The features that are input to the axial attention module, , , These represent the attention weight maps calculated along the channel axis, height axis, and width axis, respectively. This indicates an element-wise multiplication operation. This indicates a fusion convolution operation. Represents the G-type activation function. This represents the characteristics after the residuals are added and merged.

[0039] In the final output stage of the hybrid spatiotemporal feature module, the features output by the axial attention module are nonlinearly mapped through a G-type activation function to generate a spatiotemporal representation vector with high perceptuality, providing global correlation features for subsequent networks.

[0040] The structure diagram of the adaptive decoupling residual module is as follows: Figure 9As shown, the module consists of a main semantic branch and a gradient enhancement branch. This module utilizes a dual-path parallel processing mechanism to separate and extract smooth semantic features and gradient response features. The main semantic branch comprises convolutional layers, deep temporal convolutional layers, and deep feature convolutional layers. After the input features are processed through the above three paths, they are concatenated and aggregated along the channel dimension, and then input into a fusion convolution for cross-channel information interaction. The interacted features are then processed by… The gating module calculates the channel-level attention distribution and generates modulated smooth semantic features using element-wise multiplication operations. The gradient enhancement branch comprises two parallel paths: temporal difference convolution and feature difference convolution. Both paths integrate multi-scale difference operators, with internal structures as follows: Figure 10 As shown, the input features are processed by parallel extraction of gradient response components of different dimensions using staggered difference operators with strides of 1 and 2. These two components are then concatenated and aggregated before being input into the corresponding fusion convolution to perform feature integration. The integrated difference features are then processed by... The gating module generates adaptive weights and uses element-wise multiplication to generate modulated gradient enhancement features. In the feature fusion and output stage, the modulated smooth semantic features and the modulated gradient enhancement features are initially fused through an addition node. The fused features are then input to a mapping convolution to perform feature mapping, and the mapping result is then compared with the original input features through a residual jumper. After performing an addition operation and a nonlinear mapping using a G-type activation function, the final decoupling feature is output. Its mathematical expression is as follows: , , , in, Indicates input features, This represents the original smooth features after cross-channel information interaction in the main semantic branch. This represents the original difference features after fusion and convolution integration in the gradient enhancement branch. and This represents the adaptive weights generated by the gating network. This indicates an element-wise multiplication operation. This indicates a blending projection operation. This represents the activation function. This indicates the final decoupling output result.

[0041] The global context awareness module consists of a global differential attention module and a global context extraction module. The global differential attention module employs an improved multi-head self-attention mechanism, with the following structure: Figure 11 As shown. Query matrix AND key matrix Value matrix generated via linear mapping An architecture is jointly constructed using convolutional feature extraction and differential gradient enhancement. Value matrix In the generation branch, features are input to parallel grouped convolution and differencing modules, which jointly extract basic local features and spatial gradient features. The two features are then fused by additive nodes, normalized by a normalization layer, and mapped by a G-type activation function to generate a value matrix containing the differencing gradient. .matrix , , The input multi-head self-attention layer computes global dependencies, and the attention feature output sequence undergoes a nonlinear transformation via a linear layer, residual connections, and a feedforward neural network. The expression for scaled dot product attention is: , in, Represents the query matrix. Represents the key matrix. This represents a value matrix containing difference features. This represents the scaling factor of the key matrix. This represents the output of the multi-head self-attention calculation.

[0042] The global context extraction module employs a spatial pyramid pooling layer architecture. Input features are processed through multi-scale adaptive average pooling and flattening concatenation, then input to a single-layer fully connected network for dimensionality compression and feature condensation. After layer normalization, the output is a global semantic vector. The expression is as follows: , in, This represents the features output by the global differential attention module. Indicates the first Adaptive average pooling and flattening operations for different pooling kernel sizes. This represents the concatenation operation of multi-scale pooling features. and This represents the weights and biases of a single-layer fully connected network. Presentation layer normalization operation.

[0043] The semantic feedback modulation module consists of a linear mapping layer and a feature linear modulation layer. The linear mapping layer uses the global semantic vector... As input, the vector is expanded in dimension and divided into equal channels via a fully connected network, and then mapped to generate a scaling parameter vector. With bias parameter vector The feature linear modulation layer receives multiple shallow features from the main network feature extraction module at different stages. Based on the scaling and bias parameters, a residual affine modulation mechanism is executed to achieve top-down recalibration. The operation expression is as follows: in, This indicates an element-wise multiplication operation. This represents the shallow, raw features of the input. and These represent the scaled feature vector and the bias feature vector, respectively, adaptively generated by the linear mapping layer. This is an identity mapping adjustment factor used to ensure that the modulation gain is adjusted accordingly. When the underlying primitive communication behavior characteristics approach zero, they do not degenerate. This indicates the recalibration characteristics after modulation output.

[0044] The multi-task joint output module consists of a classification mapping head and a contrastive feature projection head, which are constructed in parallel. The contrastive feature projection head is composed of a fully connected dimensionality reduction layer, a batch normalization layer, a G-type activation function, and a projective fully connected layer, which are cascaded together. The output features are mapped onto a low-dimensional unit hypersphere via an L2 normalization layer to generate feature vectors. This is used to calculate the supervised contrastive loss, constraining the aggregation of feature vectors from similar few-sample devices within the feature space. The mathematical expression for L2 normalized feature projection is as follows: , in, This represents the final deep features extracted by the main network. This represents a composite projective mapping operation that includes dimensionality reduction and nonlinear activation. The L2 norm of a vector is used to divide the eigenvector by this norm. Constrained on a unit hypersphere.

[0045] The classification mapping head consists of a fully connected dimensionality reduction layer, a G-type activation function, a batch normalization layer, a random deactivation layer, and a linear classification layer, ultimately outputting the predicted probability distribution for each type of device. .

[0046] To ensure the robustness of the network and to fully extract features, this invention uses the GELU Gaussian error linear unit activation function as the G-type activation function.

[0047] Step 4, Selecting the Loss Function and Evaluation Metric: During the network training phase, the overall objective loss function adopts a weighted combination of cross-entropy loss with labeled smoothing mechanism and supervised contrastive loss. , in, This represents the total optimization loss. This represents the cross-entropy loss with labeled smoothing mechanism. Indicates the comparative loss under supervision. The weighting coefficient for the contrast loss is set to 0.1.

[0048] Labeled smoothing cross-entropy loss aims to alleviate overfitting caused by overconfidence, and its expression is: , in, The total number of categories, smoothing factor Set to 0.1. To predict probabilities, This is a real label.

[0049] The evaluation system uses accuracy, precision, recall, and the macro-average F1 score, which objectively measures classification performance under long-tailed distribution.

[0050] Accuracy reflects the model's overall ability to identify all categories of devices, and its mathematical expression is as follows: , in, This indicates the total number of IoT device categories. This represents the total number of test samples. Represents the th in the confusion matrix Line 1 The value of the column, i.e., the number of columns. The number of samples whose devices were correctly identified.

[0051] Precision reflects the proportion of samples that the model predicts as belonging to a certain class, and the percentage of samples that actually belong to that class. The accuracy expression for this type of device is as follows: , in, Indicates the first The number of samples whose devices were correctly identified. Represents the th in the confusion matrix The sum of the values ​​in the column, i.e., the model prediction of the [number]th [column]. Total number of samples for each type of device.

[0052] Recall reflects a model's ability to cover a particular class of samples. The recall rate expression for this type of device is as follows: , in, Indicates the first The number of samples whose devices were correctly identified. Represents the th in the confusion matrix The sum of the values ​​in each row, i.e., the actual label is the first row. Total number of samples for each type of device.

[0053] The macro-average F1 score, by averaging the F1 scores of each category, can objectively evaluate the overall performance of the model on an imbalanced dataset. Its expression is as follows: , in, and They represent the first Precision and recall rates for similar devices.

[0054] Step 5, train the network model: During model optimization, the global maximum number of training epochs was set to 200. The AdamW optimization algorithm was used for backpropagation parameter updates, with a weight decay coefficient of 0.005 and a base maximum learning rate. Configured as To ensure model stability during the initial training phase and avoid gradient explosion, a dynamic learning rate adjustment strategy employs cosine annealing decay with a warm-up mechanism, where the warm-up period is set to 5 rounds. The learning rate update expression under this strategy is as follows: , in, Indicates the current training step count. This represents the total number of training steps. This represents the number of warm-up steps and the minimum learning rate. Set as .

[0055] Step 6, save the model: After the training cycle ends, the system automatically extracts and solidifies the weight parameters that perform best on the validation set based on the macro-average F1 score in the evaluation framework. This operation saves the complete set of model parameters and weights, thereby determining the final IoT device identification model with generalization capabilities to meet the online inference deployment needs of real-world network scenarios.

Claims

1. A method for identifying IoT devices based on semantic perception representation and adaptive semantic feedback, characterized in that, Includes the following steps: Step 1: Select a suitable network traffic dataset for IoT devices and divide the dataset into training set, validation set and test set according to a certain ratio; Step 2: Obtain the original network traffic statistics features, group and reconstruct the features according to the preset physical semantic logic; use nonlinear transformation to balance outliers, and apply histogram equalization algorithm to enhance traffic texture differences, mapping the feature data into a pseudo-color traffic fingerprint image with physical semantic association. Step 3: Build an IoT device identification network, which consists of a feature extraction module, a global context perception module, a semantic feedback modulation module, and a multi-task joint output module. Step 4, select appropriate loss function and evaluation metrics: select a joint loss function consisting of labeled smooth cross-entropy loss function and supervised contrastive learning loss function; use accuracy, precision, recall and macro average F1 score as performance evaluation metrics for the model. Step 5: Train the IoT device identification model. Preprocess the dataset from Step 1 using the method described in Step 2, and then input the preprocessed data into the IoT device identification model built in Step 3 for training. Step 6: Solidify the model parameters based on the best performance of the evaluation indicators to determine the final IoT device identification model.

2. The IoT device identification method and system based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The semantic arrangement action in step 2 specifically involves sequentially arranging the features in an ordered manner according to the physical semantic order of time-related features, packet count features, byte stream features, packet length distribution features, arrival interval features, TCP flag features, and connection behavior features.

3. The IoT device identification method and system based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The adaptive decoupling residual module in step 3 consists of a main semantic branch and a gradient enhancement branch. The main semantic branch extracts smooth contextual semantic components through a multi-branch convolutional structure. The gradient enhancement branch forms a multi-scale difference operator group through parallel sub-branches containing asynchronous long misaligned difference operations to extract gradient change information under different receptive fields.

4. The IoT device identification method and system based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The semantic feedback modulation module in step 3 adopts a feature linear modulation mechanism that introduces an identity mapping gain: it generates scaling parameters and bias parameters based on the global semantic vector, and performs dynamic calibration on the shallow features output by the feature extraction module in the form of a residual affine transformation with a reference feature retention term.

5. The IoT device identification method based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The feature extraction module consists of a multi-scale convolution module, a dynamic gated residual module, a hybrid spatiotemporal feature module, and an adaptive decoupling residual module. The multi-scale convolution module, the dynamic gated residual module, and the hybrid spatiotemporal feature module are sequentially cascaded to capture local spatiotemporal correlation features. The adaptive decoupling residual module consists of a main semantic branch and a gradient enhancement branch. Through a dual-path parallel processing mechanism, the smooth semantic features and gradient response features are separated and extracted.

6. The IoT device identification method based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The global context perception module consists of a global differential attention module and a global context extraction module. The global differential attention module constructs global dependencies of features through a multi-head self-attention mechanism, while the global context extraction module reduces the dimensionality of features and condenses them into a global semantic vector through spatial pyramid pooling.

7. The IoT device identification method based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The semantic feedback modulation module consists of a linear mapping layer and a feature linear modulation layer. The linear mapping layer generates dynamic scaling and bias parameters based on the global semantic vector, and the feature linear modulation layer performs channel-level affine transformation on the shallow features output by the feature extraction module to achieve top-down feature recalibration.

8. The IoT device identification method based on semantic perception representation and adaptive semantic feedback according to claim 1, characterized in that, The multi-task joint output module consists of a classification mapping head and a contrastive feature projection head; the classification mapping head outputs the probability distribution of device identity through a fully connected mapping, and the contrastive feature projection head performs supervised comparison clustering through high-dimensional space mapping to optimize the feature distribution.

9. An IoT device identification system based on semantic perception representation and adaptive semantic feedback, characterized in that, Implement the method described in any one of claims 1-8.

Images