A Power Internet of Things Traffic Classification Method and System Based on Self-Attention Deep Learning

By using a CNN-RNN hybrid network structure based on self-attention deep learning, the problems of slow convergence speed and low accuracy in power IoT traffic classification are solved, achieving faster model training and higher classification accuracy, and adapting to changes in the power IoT environment.

CN117056829BActive Publication Date: 2026-03-03INFORMATION COMM COMPANY STATE GRID SHANDONG ELECTRIC POWER
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202311117785.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-08-31
Publication Date
2026-03-03
Estimated Expiration
2043-08-31

AI Technical Summary

Technical Problem

Existing power IoT traffic classification methods suffer from slow convergence, low accuracy, and poor generalization in the power IoT environment. In particular, methods based on CNN or RNN models perform poorly when faced with port confusion and random port allocation.

Method used

We employ a CNN-RNN hybrid network structure based on self-attention deep learning. By extracting spatial and channel attention features of traffic data through the self-attention mechanism, and combining them with the Softmax function for classification, we leverage the complementarity of RNN and CNN to increase the weight of important features and decrease the weight of unimportant features, thereby achieving fast convergence and high accuracy.

Benefits of technology

It accelerates the convergence speed of the model, improves the accuracy and efficiency of power IoT traffic classification, can adapt to changes in the IoT environment more quickly, and reduces training time and resource consumption.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117056829B_ABST
    Figure CN117056829B_ABST
Patent Text Reader

Abstract

This disclosure provides a method and system for traffic classification in the power Internet of Things (IoT) based on self-attention deep learning, relating to the field of power network traffic analysis technology. The method involves acquiring traffic packets in the power IoT communication environment and initially classifying them according to protocol categories. The data in each initially classified traffic packet is then parsed in binary format, converting each traffic item into a hexadecimal string. Traffic data from each category is sampled and input into a CNN-RNN model. Spatial and channel attention features of the traffic data are extracted using a self-attention mechanism. Finally, the category output is converted into a probability distribution using a Softmax function. This method accelerates model convergence and improves the accuracy of traffic data classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of power network traffic analysis technology, specifically to a power Internet of Things (IoT) traffic classification method and system based on self-attention deep learning. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] As network services and structures become increasingly complex, the challenges of network management and security defense are becoming more severe. Classifying network traffic and understanding its categories and associated applications, protocols, or services is beneficial for improving Quality of Service (QoS), facilitating security analysis, and promoting the identification of malicious behavior. Regarding traffic fragmentation in the power Internet of Things (IoT) environment, while the power system, supported by intelligent IoT, can achieve more intelligent scheduling, integrated control, and information analysis, the introduction of numerous IoT devices also brings significant security risks and process management pressures. Therefore, classifying traffic beforehand is crucial and beneficial for further analysis and processing.

[0004] Currently, traffic classification methods are mainly divided into three categories: those based on packet header information (such as ports), those based on payloads, and those based on statistical features and machine learning. Port- or payload-based classification methods are typically suitable for fixed networks and may be affected by port obfuscation, NAT, and random port allocation, making them unsuitable for the current power IoT environment. Machine learning and deep learning methods are increasingly widely used in traffic identification; however, traffic classification methods based on CNN or RNN models suffer from slow convergence, low accuracy, and poor generalization ability. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes a power IoT traffic classification method and system based on self-attention deep learning. It utilizes the complementarity of RNN and CNN to extract coarse-grained and fine-grained features from the data, and employs a self-attention mechanism to increase the weight of important features and decrease the weight of unimportant features, thereby accelerating model convergence and improving classification accuracy.

[0006] According to some embodiments, the present disclosure adopts the following technical solutions:

[0007] A power IoT traffic classification method based on self-attention deep learning includes:

[0008] Acquire traffic packets in the power Internet of Things communication environment and classify the different traffic packets according to protocol categories;

[0009] The data in each traffic packet after initial classification is parsed in binary format, and each traffic packet is converted into a hexadecimal string;

[0010] Traffic data in each type of traffic packet is sampled and input into a CNN-RNN model. Spatial attention features and channel attention features of the traffic data are extracted through a self-attention mechanism. Finally, the category output is converted into a probability distribution through a Softmax function.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions:

[0012] A power IoT traffic classification system based on self-attention deep learning includes:

[0013] The data acquisition module is used to acquire traffic packets in the power Internet of Things communication environment and to initially classify different traffic packets according to protocol categories;

[0014] The preprocessing module is used to perform binary parsing on the data in each traffic packet after initial classification, converting each traffic packet into a hexadecimal string;

[0015] The classification module is used to sample traffic data in each type of traffic packet and input it into the CNN-RNN model. It extracts the spatial attention features and channel attention features of the traffic data through the self-attention mechanism, and finally converts the category output into a probability distribution through the Softmax function.

[0016] According to some embodiments, the present disclosure adopts the following technical solutions:

[0017] A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the power Internet of Things traffic classification method based on self-attention deep learning.

[0018] According to some embodiments, the present disclosure adopts the following technical solutions:

[0019] A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the power Internet of Things traffic classification method based on self-attention deep learning.

[0020] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0021] This disclosure provides a power Internet of Things (IoT) traffic classification method based on self-attention deep learning. It uses a hybrid network structure of CNN and RNN based on self-attention mechanism for traffic classification, and treats traffic classification as traditional image classification, processing the training data into pseudo-image form. It utilizes the complementary nature of RNN and CNN to extract coarse and fine-grained features of the data, and uses the self-attention mechanism to increase the weight of important features and decrease the weight of unimportant features, thereby accelerating the model convergence speed and improving the classification accuracy.

[0022] This disclosure first uses the DataLoader class in the PyTorch framework to load the dataset and generate iterable training and test datasets. To address the issue of large differences in the amount of training data for different categories, when loading hexadecimal strings, it checks if the number exceeds a threshold; if it does, random sampling is performed according to the threshold. Model training uses the Adam optimizer and cross-entropy function, with a learning rate set to 0.001. Then, the training data is fed into the model for each iteration, training for 20 epochs. Every two epochs, the model's performance is tested using the test dataset, with accuracy, precision, recall, and F1 score as evaluation metrics. An early stopping point is set; when the loss on the validation set stops changing or even increases after multiple iterations, the model is considered converged, and training can be stopped early. The model's structure and parameters are saved for later direct loading without needing to train a new model. When the IoT environment changes or the dataset is different, the previously saved model can be used as a pre-trained model to train a suitable model faster in the updated environment, improving the convergence speed and classification efficiency of the model training. Attached Figure Description

[0023] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0024] Figure 1 This is a flowchart of an embodiment of the present disclosure.

[0025] Figure 2 This is a flowchart illustrating the data acquisition and preprocessing process in an embodiment of this disclosure.

[0026] Figure 3 A diagram of a CNN-RNN model with a self-attention mechanism added to an embodiment of this disclosure.

[0027] Figure 4 The performance of the CNN-RNN model with self-attention mechanism added to the embodiments of this disclosure is evaluated.

[0028] Figure 5 The performance of the CNN-RNN model without attention mechanism in the embodiments of this disclosure is evaluated. Detailed Implementation

[0029] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] Example 1

[0033] One embodiment of this disclosure provides a power Internet of Things (IoT) traffic classification method based on self-attention deep learning, including:

[0034] Step 1: Obtain traffic packets in the power Internet of Things communication environment and perform preliminary classification of different traffic packets according to protocol type;

[0035] Step 2: Perform binary parsing on the data in each traffic packet after initial classification, converting each traffic packet into a hexadecimal string;

[0036] Step 3: Sample the traffic data in each type of traffic packet and input it into the CNN-RNN model. Extract the spatial attention features and channel attention features of the traffic data through the self-attention mechanism. Finally, convert the category output into a probability distribution through the Softmax function.

[0037] As one embodiment, a specific implementation of the power Internet of Things traffic classification method based on self-attention deep learning disclosed herein includes:

[0038] Step 1: Obtain traffic packets in the power IoT communication environment, perform preliminary classification of different traffic packets according to protocol type, and preprocess the data in the traffic packets;

[0039] Specifically, a Modbus client was created using the Python Modbus library, and then the simulation software ModbusRSSim was used as a slave to communicate with the client created by PyModbus. For MQTT protocol packets, an emqx server was used as the MQTT server, deployed on the management host, and scripts from the Python paho.mqtt library were used for subscription and sending, interacting with the MQTT broker on Windows. During communication, Wireshark was used to capture packets on the corresponding ports, and ARP, DHCP, and other data irrelevant to this experiment were removed. Then, the traffic packets were categorized according to different protocol types.

[0040] The data preprocessing stage involves binary parsing of the categorized traffic packets, converting each traffic entry in the pcap packet into a hexadecimal string, and saving them in a file according to their category. The goal is to convert the traffic data into a pseudo-image format so that feature extraction can be performed using models such as CNNs and RNNs.

[0041] Step 2: Sample the traffic data from each type of traffic packet and input it into the CNN-RNN model for classification. This includes:

[0042] 1. Network Model Construction

[0043] CNN-RNN models sometimes neglect learning important features or spend excessive resources extracting minor features. To address this issue, this disclosure proposes a CNN-RNN network traffic classification model based on a self-attention mechanism. The structure of the CNN-RNN model includes: convolutional layers, normalization layers, self-attention modules, pooling layers, LSTM layers, fully connected layers, softmax layers, and Dropout.

[0044] The specific structure is as follows: The CNN-RNN model uses the ReLU activation function. The first layer is a one-dimensional convolutional layer with 1 input channel, 32 output channels, a kernel size of 3, and a stride of 1. The second layer is a batch normalization layer. The third layer is a self-attention module used to extract spatial and channel attention features of the input. The fourth layer is a max pooling layer with a kernel size of 3 and a stride of 1. The first four layers are then repeated. The ninth layer is an LSTM layer with an input size of sequence_length - 4 * 2, an output size of 256, and a dropout of 0.5. The tenth layer is another LSTM layer with an input size of 256, an output size of 100, and a dropout of 0.5. The eleventh layer is a fully connected layer with an input size of 100 and an output size of 108. The twelfth layer is a fully connected layer with an input size of 108 and an output size equal to the total number of classes. The last layer is a Softmax layer used to convert the output into a probability distribution. The model's Dropout layer is used to randomly drop some neurons during training to avoid overfitting. This model is built using the PyTorch framework.

[0045] 2. Model Training and Evaluation

[0046] This CNN-RNN model first uses the DataLoader class in the PyTorch framework to load the dataset, generating iterable training and testing datasets. To address the issue of large differences in the amount of training data for different categories, when loading hexadecimal strings, it checks if the number exceeds a threshold; if it does, random sampling is performed according to the threshold. Model training uses the Adam optimizer and cross-entropy function, with a learning rate set to 0.001. The training data is then fed into the model for 20 iterations each time, and the model's performance is tested every two iterations using the test dataset. Four evaluation metrics are used: accuracy, precision, recall, and F1 score. An early stopping point is set; when the loss on the validation set no longer changes or even increases after multiple iterations, the model is considered converged, and training can be stopped early. The model's structure and parameters are saved for later direct loading without needing to train a new model. When the IoT environment changes or the dataset is different, the previously saved model can be used as a pre-trained model to train a suitable model more quickly in the updated environment.

[0047] In the experiments disclosed herein, four quantitative metrics were used to evaluate the model's performance: accuracy, precision, recall, and F1 score. Accuracy is an important metric for evaluating overall classification performance, reflecting the model's classification effectiveness in practical applications. However, accuracy evaluation may be problematic when sample classes are imbalanced. In such cases, precision and recall are needed, which are often contradictory evaluation metrics. To comprehensively evaluate the classifier's prediction performance, the F1 score, a weighted harmonic mean of precision and recall, can be used. A higher F1 score indicates better model classification performance. In the experiments, the scikit-learn machine learning library was used to calculate these four metrics. To accurately evaluate the system's performance, this technical solution measured all evaluation metrics 10 times in the experiments and took the average of the ten experiments.

[0048] The specific steps of the experiment are as follows: First, the dataset needs to be converted into a format that the model can accept. First, the previously saved hexadecimal string of traffic data is loaded using the panda library, and then converted into a PyTorch tensor type. Next, the TensorDataset and DataLoader functions from the torch library are used to convert the tensor data into iterable training and test sets. Finally, the model is trained using the training set, and the classification performance is tested using the test dataset in specified rounds. Ten experiments are conducted, and the average of the ten experiments in rounds 1, 3, 5, and 8 is recorded. The experimental results are as follows: Figure 4 As shown, an accuracy of 0.953 and an F1 score of 0.943 were obtained. To demonstrate the effectiveness of the self-attention mechanism, it was compared with a CNN-RNN model without an attention mechanism in a unified environment using the same dataset. Ten experiments were conducted, and the average value was taken. Figure 5 As shown, an accuracy of 0.928 can be obtained, which is about two percentage points lower than the model with self-attention mechanism, but from... Figure 4 Figure 5 The comparison shows that the model with self-attention converges faster. The model with self-attention converges by the 5th epoch, while the model without self-attention converges around the 13th epoch.

[0049] Next, let's examine the specific performance of the classification. The CNN-RNN hybrid model based on a self-attention mechanism was evaluated on an IoT traffic classification test set, generating a row-normalized confusion matrix. The rows of this confusion matrix represent the actual class of the sample, and the columns represent the predicted label; therefore, the matrix is ​​row-normalized. The main diagonal elements indicate that the model can classify each traffic segment without causing too much confusion. However, by combining the accuracy analysis of each class using the scikit-learn machine learning library, some interesting confusion situations can be observed. Protocols 2 and 6, namely modbus_func_3 and modbus_func_4, appear frequently. Their actual functions are reading the holding register and reading the input register, respectively. Due to their similar functions, and after actual analysis of the specific message details, the Query message only differs by a few bytes such as the function code. Therefore, for the classification model, there is a certain probability of confusion between the two protocols. Although confusion is possible, the CNN-RNN hybrid model based on the self-attention mechanism can still capture key features well for classification, achieving an accuracy of around 90% for both categories.

[0050] In addition to the dataset collected in a simulated IoT environment, two commonly used public datasets, UNSW-NB15 and NSL-KDD, were used to evaluate the CNN-RNN hybrid model scheme based on the self-attention mechanism. The NSL-KDD dataset, provided by the University of New Brunswick, is a commonly used dataset in the field of network intrusion detection and is widely used for performance evaluation and comparison of intrusion detection algorithms. This dataset is an upgraded version of KDD Cup 1999. Through preprocessing and feature selection of the original dataset, some redundant and duplicate samples and features were removed, and some new network intrusion scenarios were added, making the dataset more suitable for real-world network security problems. It contains 23 attack categories and is a multi-class dataset. The UNSW-NB15 dataset is a network intrusion detection dataset provided by the Network and Systems Security Research Group at the University of Technology Sydney. This dataset contains intrusion behaviors captured from real network traffic, including normal network traffic and network traffic with attack behaviors, and is used here as a binary classification dataset. Both datasets include detailed features for each connection, such as source IP address, destination IP address, source port, destination port, protocol type, etc. Following the previous experimental principles, 10 repeated experiments were conducted, and the average value was recorded. On the NSL-KDD dataset, the accuracy reached approximately 0.868; on the UNSW-NB15 dataset, the accuracy reached approximately 0.850. Since these two public datasets use plaintext data and have many features, the model can quickly and effectively learn relevant features for classification. The experimental results show that the model converged in the second epoch on both datasets, similar to previous experiments on IoT protocol datasets. In terms of training and test accuracy, the training accuracy was higher than the test accuracy and tended to stabilize after the second epoch. This is because the model had already learned all the features, leading to overfitting to the training data.

[0051] 3. The constructed CNN-RNN model is used to classify traffic. This model can automatically learn and extract traffic features, and with the support of the self-attention mechanism, it can converge faster and achieve higher accuracy.

[0052] Specifically, traffic data in each type of traffic packet is sampled and input into the CNN-RNN model. Spatial attention features and channel attention features of the traffic data are extracted through the self-attention mechanism. Finally, the category output is converted into a probability distribution through the Softmax function.

[0053] The sampling of traffic data in each type of traffic packet includes:

[0054] When loading hexadecimal strings, the amount of traffic data in the same category of traffic packets is judged to determine whether the amount of traffic data in a single category is higher than a threshold. If it is higher than the threshold, the traffic data in that category is sampled and input into the CNN-RNN model to prevent the amount of data in different categories from being too different.

[0055] After sampling, the data is input into a CNN-RNN model. Spatial and channel attention features of the traffic data are extracted using a self-attention mechanism. Finally, the softmax function is used to convert the category output into a probability distribution; including:

[0056] The detailed structure of the self-attention mechanism is as follows: In the module's initialization method, several convolutional and pooling layers are defined. First, an adaptive average pooling layer (nn.AdaptiveAvgPool1d) is used to pool the input, followed by an adaptive max pooling layer (nn.AdaptiveMaxPool1d) for further pooling. Next, two convolutional layers are used for feature extraction and dimensionality reduction of the input. The first convolutional layer has the same number of output channels as the input channels (1 / reduction), while the second convolutional layer has the same number of output channels as the input channels. Finally, two sigmoid activation functions are defined to map the output values ​​of the convolutional layers to the range [0, 1], representing the importance of the feature maps. Specifically, the self-attention mechanism module receives an input tensor x. It first extracts channel features through adaptive average pooling and adaptive max pooling layers, then performs feature scaling through two fully connected layers. The sigmoid activation function is used to limit the output to between 0 and 1, serving as the weight for channel feature scaling. Finally, the scaled channel feature result is multiplied by the original input to obtain the channel feature selection result. Next, the self-attention mechanism module extracts spatial attention features and performs feature selection in the spatial dimension. First, it performs average pooling and max pooling on the input. Then, it concatenates the two results in the channel dimension, reduces the number of channels from 64 to 1 using a 1x1 convolutional layer, and uses a sigmoid activation function to constrain the output to between 0 and 1, serving as weights for spatial feature scaling. Finally, it multiplies the channel feature scaling and spatial feature scaling results to obtain the output of the self-attention mechanism module.

[0057] Finally, the Softmax function outputs a probability vector for each class. In multi-class classification problems, the Softmax function transforms a set of real values ​​into a vector representing a probability distribution. The Softmax function converts the original output into a probability distribution, allowing the output for each class to be interpreted as the probability of that class. This facilitates prediction and decision-making in multi-class classification problems.

[0058] Example 2

[0059] One embodiment of this disclosure provides a power Internet of Things (IoT) traffic classification system based on self-attention deep learning, comprising:

[0060] The data acquisition module is used to acquire traffic packets in the power Internet of Things communication environment and to initially classify different traffic packets according to protocol categories;

[0061] The preprocessing module is used to perform binary parsing on the data in each traffic packet after initial classification, converting each traffic packet into a hexadecimal string;

[0062] The classification module is used to sample traffic data in each type of traffic packet and input it into the CNN-RNN model. It extracts the spatial attention features and channel attention features of the traffic data through the self-attention mechanism, and finally converts the category output into a probability distribution through the Softmax function.

[0063] Example 3

[0064] One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the power Internet of Things traffic classification method based on self-attention deep learning.

[0065] Example 4

[0066] One embodiment of this disclosure provides an electronic device, including a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the power Internet of Things traffic classification method based on self-attention deep learning.

[0067] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0069] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A power IoT traffic classification method based on self-attention deep learning, characterized in that, include: Acquire traffic packets in the power Internet of Things communication environment and classify the different traffic packets according to protocol categories; The data in each traffic packet after initial classification is parsed in binary format, and each traffic packet is converted into a hexadecimal string; Traffic data in each type of traffic packet is sampled and input into the CNN-RNN model. Spatial attention features and channel attention features of the traffic data are extracted through the self-attention mechanism. Finally, the category output is converted into a probability distribution through the Softmax function. Traffic packets in the power Internet of Things (IoT) communication environment are acquired, and different traffic packets are initially classified according to protocol type, including: A Modbus client was created using the Python Modbus library, and then the simulation software ModbusRSSim was used as a slave to communicate with the client created by Pymodbus. For MQTT protocol packets, an emqx server was used as the MQTT server, deployed on the management host, and scripts from the Python paho.mqtt library were used for subscription and sending, interacting with the MQTT broker on Windows. During communication, Wireshark was used to capture packets on the corresponding ports and remove data unrelated to ARP and DHCP, and then the traffic packets were initially classified according to different protocol categories.

2. The power IoT traffic classification method based on self-attention deep learning as described in claim 1, characterized in that, The data in each traffic packet after initial classification is parsed in binary format. Each traffic item in each traffic packet is converted into a pseudo-image form of a hexadecimal string and saved in a file according to the category.

3. The power IoT traffic classification method based on self-attention deep learning as described in claim 1, characterized in that, Traffic data is sampled from each type of traffic packet, including: When loading a hexadecimal string, the amount of traffic data in the same category of traffic packets is judged to determine whether the amount of traffic data in a single category is higher than a threshold. If it is higher than the threshold, the traffic data in the category is sampled and input into the CNN-RNN model.

4. The power IoT traffic classification method based on self-attention deep learning as described in claim 1, characterized in that, The structure of the CNN-RNN model includes: convolutional layers, normalization layers, self-attention modules, pooling layers, LSTM layers, fully connected layers, softmax layers, and Dropout.

5. The power IoT traffic classification method based on self-attention deep learning as described in claim 1, characterized in that, In the self-attention mechanism, after receiving the input tensor, channel attention features are first extracted through adaptive average pooling and adaptive max pooling layers. Then, the features are scaled through two fully connected layers. An activation function is used to limit the output to between 0 and 1 as the weights for scaling the channel attention features. Finally, the scaled channel attention features are multiplied by the original input tensor to obtain the result after channel attention feature selection.

6. The power IoT traffic classification method based on self-attention deep learning as described in claim 5, characterized in that, Next, the self-attention mechanism extracts spatial attention features and performs feature selection in the spatial dimension. First, the input is subjected to average pooling and max pooling. Then, the two results are concatenated in the channel dimension. The number of channels is reduced from 64 to 1 through a 1x1 convolutional layer, and the output is restricted to between 0 and 1 using the Sigmoid activation function as a weight for spatial feature scaling. Finally, the results of channel attention feature scaling and spatial attention feature scaling are multiplied to obtain the output of the self-attention mechanism.

7. A power IoT traffic classification system based on self-attention deep learning, employing any one of the power IoT traffic classification methods based on self-attention deep learning according to claims 1-6, characterized in that, include: The data acquisition module is used to acquire traffic packets in the power Internet of Things communication environment and to initially classify different traffic packets according to protocol categories; The preprocessing module is used to perform binary parsing on the data in each traffic packet after initial classification, converting each traffic packet into a hexadecimal string; The classification module is used to sample traffic data in each type of traffic packet and input it into the CNN-RNN model. It extracts the spatial attention features and channel attention features of the traffic data through the self-attention mechanism, and finally converts the category output into a probability distribution through the Softmax function.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the power Internet of Things traffic classification method based on self-attention deep learning as described in any one of claims 1-6.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the power Internet of Things traffic classification method based on self-attention deep learning as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Network traffic classification method and system based on multi-scale feature attention

    CN112839024A

  • Small sample radar one-dimensional image target identification method based on deep transfer learning

    CN114488140A

  • Network attack classification method and system for power distribution network

    CN115879030A