Abnormal flow detection method and device based on device type

Abstract

The present invention discloses a device-type-based abnormal traffic detection method and apparatus, comprising: establishing a mapping relationship between a device MAC address and a device type; extracting the device MAC address from the traffic to be detected, and selecting an abnormal traffic detection model for the corresponding device type based on the mapping relationship; and inputting the traffic characteristics of the traffic to be detected into the abnormal traffic detection model for the corresponding device type to obtain an abnormal traffic detection result. The present invention can automatically identify the device type generating the traffic and input it into the abnormal traffic detection model bound to its device type for detection, thereby more promptly and accurately detecting the attack status of devices in the network and issuing an alarm.

Description

Technical Field

[0001] The present invention relates to the field of flow detection, and in particular to a method and device for detecting abnormal flow based on device type. Background Art

[0002] Abnormal traffic detection, as an effective defense measure, monitors the inbound and outbound traffic of a device to detect attacks. Compared to vulnerability detection and signature-based intrusion detection, it offers the advantages of low deployment costs and timely detection of unknown attacks. Current abnormal traffic detection methods primarily extract traffic features and then input these features into a model to distinguish between abnormal and normal traffic. Based on the method used, abnormal traffic detection can be categorized into four main types: classification-based, statistics-based, machine learning-based, and deep learning-based. With the advancement of machine learning and deep learning, these two methods have become widely used algorithms for abnormal traffic detection.

[0003] Yisroel Mirsky and his colleagues (Mirsky Y, Doitshman T, Elovici Y, et al. Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection [C] / / Network and Distributed System Security Symposium, 2018) introduced Kitsune, a plug-and-play NIDS. This NIDS uses an unsupervised anomaly detection model constructed based on autoencoders to detect and monitor IDS models online, improving the accuracy of unsupervised anomaly detection models. Thien Duc Nguyen (Nguyen TD, Marchal S, Miettinen M, et al. D "IoT: A Federated Self-learning Anomaly Detection System for IoT" [J]. 2018.) proposed an autonomous, self-learning distributed system that can effectively detect infected IoT devices. DIOT automatically identifies device types and uses federated learning to establish a normal traffic model for detecting anomaly traffic, reducing the false positive rate of the anomaly traffic detection model. Security company NSFOCUS Technology proposed the IoT Access Gateway IoT-AG to protect and expand the security of IoT devices. It uses active device detection and discovery technology, combined with machine learning algorithms and access relationship whitelists, to manage and control the behavior of connected devices. Security company Qi'anxin proposed an IoT access control system to address secure access management issues in scenarios where a large number of IoT devices of various types are used within a network. It implements security management functions through modules such as IoT device discovery and identification, access awareness and user identification, unified access control for multiple device types, counterfeit detection and disposal, security compliance checks, status monitoring, and IP address management and usage monitoring.

[0004] However, current research on abnormal traffic detection technology does not differentiate between the types of devices generating traffic when training models, attempting to train a single, broadly applicable abnormal traffic detection model. With the proliferation of device types, hundreds of them have flooded the market, posing several challenges to existing abnormal traffic detection solutions.

[0005] First, it becomes more difficult to collect training data. To ensure effective model training, the abnormal traffic detection model must cover normal traffic from all device types. Collecting traffic requires purchasing equipment, which is expensive. Furthermore, it is not possible to purchase all types of equipment and collect all normal traffic patterns.

[0006] Secondly, the cost of model updates increases. With the rapid pace of device updates, traffic models are subject to new changes. This necessitates real-time updates to the abnormal traffic detection model to ensure effectiveness, increasing the frequency of model updates. Furthermore, to ensure comprehensiveness, training sets are often very large, resulting in significant training costs. Both the increased update frequency and the high cost of model training increase the cost of model updates.

[0007] Finally, the model's recognition accuracy decreases. Because the abnormal traffic detection model is not trained specifically, the traffic patterns of various device types vary significantly. Aggregating these data for learning affects the effectiveness of the machine learning method, causing the anomaly detection model to be unable to detect even small changes in traffic, resulting in a decrease in model detection accuracy.

[0008] Although some studies, such as Green Alliance Technology's IoT access gateway IoT-AG technology and Thien DucNguyen et al., have begun to combine device type and abnormal traffic detection, there are currently some defects.

[0009] First, the methods for identifying device types are not automated. Device types require manual labeling, or the proposed device type methods are difficult to collect device fingerprints, making them difficult to apply on a large scale. Green Alliance Technology's IoT-AG IoT access gateway and the device identification technology proposed by Thien Duc Nguyen et al. only identify IoT devices, resulting in limited coverage. Furthermore, both utilize clustering methods, requiring the purchase of appropriate equipment to collect device fingerprints, which is both costly and difficult.

[0010] Secondly, abnormal traffic detection methods are not applicable. Current abnormal traffic detection research still focuses on supervised abnormal traffic detection methods, and model training sets must include abnormal traffic. However, there are hundreds of existing device types, and they are updated rapidly. Most device types do not have dedicated security personnel to study them, and no related security issues have been exposed. Abnormal traffic cannot be collected. Supervised abnormal traffic detection methods such as Naive Bayes, Random Forest, Decision Boost, XGBoost, RNN, DNN, CNN, etc. are no longer applicable. The unsupervised detection technology proposed by Thien Duc Nguyen et al. requires specialized models for each device category, making it unsuitable for general use. Summary of the Invention

[0011] In response to the above problems, the present invention provides a method and apparatus for detecting abnormal traffic based on device type. When users use the method provided by the present invention, they can automatically identify the type of device generating traffic and input it into the abnormal traffic detection model bound to its device type for detection, thereby more promptly and accurately discovering the status of devices in the network being attacked and issuing an alarm.

[0012] The technical solution of the present invention includes:

[0013] A method for detecting abnormal traffic based on device type, comprising the following steps:

[0014] Build a mapping relationship between device MAC address and device type;

[0015] Extract the device MAC address in the traffic to be detected, and based on the mapping relationship, select the abnormal traffic detection model of the corresponding device type, wherein the abnormal traffic detection model is obtained by the following steps:

[0016] The normal traffic training set of the corresponding device type is used as a training sample;

[0017] The l used by the sparse autoencoder i The paradigm parameters are encoded as chromosomes in the genetic space, and the loss value of the sparse autoencoder training process is used as the fitness;

[0018] Initialize the population size and the initial chromosomes in the population;

[0019] Calculate each chromosome The fitness when the paradigm parameter is Based on chromosome reproductive chromosomes Where i is the chromosome number, t is the number of reproduction, t≤T, T is the number of reproduction;

[0020] According to the minimum fitness Obtaining the optimal paradigm parameters and regularizing the sparse autoencoder based on the optimal paradigm parameters to obtain an abnormal traffic detection model;

[0021] The flow characteristics of the flow to be detected are input into an abnormal flow detection model of a corresponding device type to obtain an abnormal flow detection result.

[0022] Furthermore, the method for obtaining the traffic to be detected includes: using wirshark or tcpdump to capture the ingress and egress traffic of the designated LAN.

[0023] Furthermore, the mapping relationship is constructed by the following steps:

[0024] 1) Extract device identification features of traffic samples of each device type;

[0025] 2) Based on the device recognition features and using the multi-view wide-depth algorithm, passive recognition results are obtained;

[0026] 3) Use nmap to actively scan traffic samples of each device type to obtain active identification results;

[0027] 4) Based on the passive identification results and the active identification results, the mapping relationship between the device MAC address and the device type is obtained;

[0028] Furthermore, the device identification features include: broadcast packet features and http traffic features.

[0029] Furthermore, the broadcast packet characteristics include: data packet characteristics of the DHCP protocol, the MDNS protocol, the SSDP protocol, the LLMNR protocol and the NBNS protocol.

[0030] Furthermore, the http traffic characteristics include: a user-agent field.

[0031] Furthermore, the loss value includes: mean square error.

[0032] Furthermore, the reproduction method includes at least one of: calculating the probability of being a parent based on fitness, pairing chromosomes to exchange genes, and chromosome mutation.

[0033] Furthermore, the loss function for training the sparse autoencoder is Where n is the dimension of the sparse autoencoder input vector, x p is the p-th dimension value of the input vector, is the p-th dimension value of the output vector, m is the number of neurons in the middle layer of the sparse autoencoder, λ is the optimal paradigm parameter, and w q is the weight of the qth neuron in the middle layer.

[0034] Furthermore, traffic characteristics include: packet size, flow duration, TTL, window, flag statistics, and TCP statistics of the same source IP and destination IP within a fixed time.

[0035] Furthermore, the traffic features are pre-processed before being input into the abnormal traffic detection model of the corresponding device type.

[0036] Furthermore, the preprocessing method includes: normalizing the flow characteristics.

[0037] Furthermore, when the abnormal traffic detection method is applicable to multiple local area networks, the abnormal traffic detection model is trained by a federated learning method, including:

[0038] Each training participant u obtains the abnormal traffic detection model of device type k from the security service center Or the abnormal traffic detection model parameters of device type k

[0039] Based on abnormal traffic detection model Or abnormal traffic detection model parameters Use the local normal traffic training set for training to obtain the abnormal traffic detection model Where v is the training round;

[0040] Abnormal traffic detection model Parameters Upload to the security service center;

[0041] Security Service Center based on parameters Obtaining anomaly traffic detection models and abnormal traffic detection model parameters

[0042] When the training round v reaches the set training round or abnormal traffic detection model The loss value is less than the set value, the abnormal flow detection model As a trained device type k abnormal traffic detection model.

[0043] Furthermore, perform the following steps to migrate the abnormal traffic detection model to the new device type:

[0044] 1) Use the word2vec method to process the original device type and the new device type into word vectors respectively;

[0045] 2) Based on the Euclidean distance between word vectors, obtain the original device type that is most similar to the new device type;

[0046] 3) Using the training data of the new device type, the abnormal traffic detection model of the most similar original device type is trained to obtain the abnormal traffic detection model of the new device type.

[0047] A storage medium stores a computer program, wherein the computer program is configured to execute the above method when running.

[0048] An electronic device includes a memory and a processor, wherein the memory stores a program for executing the above method.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] 1. Binding device types to abnormal traffic models improves the accuracy of abnormal traffic detection models and enhances the scalability of the abnormal traffic detection model library. Furthermore, the active and passive device identification method proposed in this paper can identify not only IoT devices but also device types such as mobile phone models and computer models. This covers a much larger area than previous IoT device identification work.

[0051] 2. Abnormal traffic detection is performed based on a sparse autoencoder, and a genetic algorithm is used to find the optimal L1 normal form parameters of the sparse autoencoder. During the training process, more important features are learned and the deep structure between features is explored, which greatly improves the accuracy of the autoencoder.

[0052] 3. Using federated learning to jointly learn with other local area networks without sharing data, the pre-set abnormal traffic detection model improves detection rates across various traffic categories compared to learning from a single local area network. This invention is the first unsupervised federated learning algorithm to combine federated learning with sparse autoencoders.

[0053] 4. Applying the idea of transfer learning to abnormal traffic detection of networked devices can enable new devices to obtain better models through fewer iterations during the training process, reducing the cost of generating new models. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Flowchart of training for abnormal traffic detection method based on device type.

[0055] Figure 2 Flowchart of the abnormal traffic detection method based on device type.

[0056] Figure 3 Schematic diagram of the network structure used by the abnormal traffic detection model.

[0057] Figure 4 Schematic diagram of the federated learning process. DETAILED DESCRIPTION

[0058] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only specific embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] The present invention can be integrated into traffic inlets and outlets such as firewalls, WiFi controllers, routers, and gateway servers to realize traffic anomaly detection. It mainly includes a traffic monitoring module, a device type identification module, an abnormal traffic detection module, and a federated learning module.

[0060] 1. Traffic monitoring module

[0061] The traffic monitoring module uses wirshark or tcpdump for monitoring, which is used to collect traffic data and input it into the subsequent device type identification module and abnormal traffic detection module for device identification and abnormal traffic detection.

[0062] 2. Device type identification module

[0063] This module mainly uses a combination of active and passive methods to identify device types.

[0064] This module first uses device communication traffic characteristics to passively identify the device type. These characteristics primarily include broadcast packet characteristics and HTTP traffic characteristics. Broadcast packet characteristics primarily include data packets from protocols such as DHCP, MDNS, SSDP, LLMNR, and NBNS. HTTP traffic characteristics primarily include fields such as the user-agent field. For example, when a Redmi Note 7 Pro accesses the WeChat login page, the user-agent field stores the phone's model.

[0065] The passive recognition method mainly uses the multi-view wide-depth algorithm proposed by (Yu L, Luo B, Ma J, et al. You are what you broadcast: identification of mobile and IoT devices from (public) WiFi. 2020.) First, onehot encoding is used for key-value pair related features. For pseudo-natural language type feature information, word2vec is used to vectorize it. All features are concatenated into a vector feature vector e(e1,e 2, e3,...,e6), a total of 6 views, which are then input into the multi-view wide-depth algorithm. The multi-view wide-depth algorithm consists of two structures:

[0066] One is a deep neural network structure to improve the generalization performance of the device recognition model. Its input is e and its output is is the classification judgment probability obtained by the deep fusion neural network, and the t c A certain type of device information, namely manufacturer, device type or device model.

[0067] The other is a breadth neural network, which is used to improve the interactive memory between device information and each feature view, that is, how each feature view responds to the manufacturer, device type and device model, which is divided into 6 views and inputs e1, e2,...,e i , the output is , is the classification judgment probability obtained by the breadth fusion neural network, the t c A certain type of device information, namely manufacturer, device type or device model.

[0068] The probability that the final device belongs to a certain category is obtained by the following formula:

[0069]

[0070] Among them, the is the classification judgment probability obtained by the breadth fusion neural network, is the classification judgment probability obtained by the deep fusion neural network, and the t c A certain type of device information, namely manufacturer, device type or device model.

[0071] Then use nmap to perform active scanning, using commands such as nmap -o 192.168.10.0 / 24 to further obtain and improve the device's operating system and device type related information.

[0072] Finally, the results of the passive identification method and active scanning are combined to generate a mapping relationship between the device MAC address and the device type.

[0073] This module can identify device types including computers, mobile phones, IoT devices, etc. It can identify devices based on four dimensions: device brand, device type, device model, and device operating system. For example, the device brand of Huawei P30 Pro is Huawei, the device type is mobile phone, the device model is P30, and the device operating system is Android.

[0074] The device identification technology device in this module uses a preset neural network model, and trains the device identification model by pre-collecting a training sample set, which includes a large number of pre-labeled feature information training samples.

[0075] This module is run before abnormal traffic detection begins. When network traffic passes through this module, a mapping between MAC addresses and device types is generated. For example, the device type of the MAC address 7c:67:a2:90:04:6b is D-Link Camera 930L.

[0076] 3. Anomaly Detection Module

[0077] This module uses the device information marked by the device identification module to input the flow characteristics into the preset abnormal flow detection and identification model of the corresponding category to determine whether it is abnormal flow.

[0078] The feature extraction component is responsible for parsing the raw binary files collected by the traffic monitoring module, reassembling the data streams, and sending the traffic flow characteristics to the anomaly detection module. The extracted traffic features mainly include packet size, flow duration, TTL, window, flag statistics, and TCP statistics for the same source and destination IP addresses within a fixed time period, totaling 85 dimensions.

[0079] The data processing part is responsible for normalizing the feature data to eliminate the influence of different measurement units. The normalization formula used in this module is as follows.

[0080]

[0081] where x max is the maximum value in data x, x min is the minimum value in data x, where X is the normalized data.

[0082] The abnormal traffic detection part uses a pre-built deep sparse autoencoder model, which is trained by pre-collected normal traffic training sample sets of specific categories. The input is the processed traffic feature data, which can be represented as a vector X(X1,X2,...,X n ), n is the dimension of the input vector, and the output is whether it is an anomaly.

[0083] An autoencoder is an unsupervised neural network algorithm. The number of input neurons is equal to the number of output neurons. The goal of training the autoencoder is to minimize the gap between input and output. The loss value calculation formula used in this module is the mean square error.

[0084]

[0085] where x p Represents the p-th dimension value of the input vector, represents the i-th dimension of the output vector, n represents the dimension of the input vector, and MSE represents the mean squared error. This module uses a 5-layer sparse autoencoder with 85 neurons in the input layer, 59 neurons in the encoder hidden layer, 42 neurons in the middle layer, 59 neurons in the decoder hidden layer, and 85 neurons in the output layer.

[0086] The sparsity of the autoencoder can be explained as follows: if the output of a neuron is close to 1, the present invention considers it to be activated, and if the output is close to 0, it is considered to be inhibited. Under normal circumstances, most neurons are inhibited. This module uses a sparse autoencoder to ensure that the autoencoder can learn deeper feature structures. The present invention uses the l1 paradigm to regularize the autoencoder, and the loss function formula is updated to:

[0087]

[0088] where w q is the weight of the qth neuron in the middle layer, m is the number of neurons in the middle layer, and λ is the paradigm parameter, which needs to be determined by the user.

[0089] To better determine the impact of the L1 normal form parameter λ on the model, this module uses a genetic algorithm to optimize the L1 normal form parameter of the sparse autoencoder. A genetic algorithm is a method that searches for optimal solutions by simulating the natural evolutionary process. This algorithm transforms the problem-solving process into something similar to the crossover and mutation of chromosome genes in natural biological evolution.

[0090] This module encodes the sparse parameters to be solved into binary form, which serves as the chromosomes of the genetic space. It also uses the MSE value during sparse autoencoder training as the fitness value, aiming to minimize the fitness value and find the optimal sparse parameters. The genetic algorithm solution process is as follows.

[0091] 1) Initialization: Initialize the initial population size and the initial chromosomes in the population.

[0092] 2) Calculate the fitness. Calculate the MSE during sparse autoencoder training using each chromosome in the population as λ.

[0093] 3) Selection calculation. Use the roulette wheel algorithm to calculate the probability of each chromosome in this round being the parent of the next generation. The smaller the fitness, the greater the probability of being selected. The probability calculation formula is

[0094]

[0095] where k i is the i-th chromosome, p(k i ) is k i The probability of being selected, is λ and k i The training mean square error is , and J is the number of chromosomes.

[0096] 4) Crossing over: The two paired chromosomes exchange genes at two fixed positions, thus forming two new chromosomes.

[0097] 5) Mutation: A fixed position on a chromosome mutates with a probability of 66%, thus forming a new chromosome.

[0098] 6) Determine whether the number of reproduction times has been reached. If so, output the lambda with the minimum fitness and end. If not, input the new population generated by 3), 4), and 5) and return to 2).

[0099] The above process can be used to obtain the local optimal λ of the sparse autoencoder.

[0100] Then use λ as l i The parameters of the paradigm are trained in a sparse autoencoder to determine the anomaly data threshold and obtain an abnormal traffic detection model. During the training phase, this module uses normal traffic data from the same device type to train an abnormal traffic detection model specifically tied to the device type. During the detection phase, the traffic characteristics of a specific device type are input into the corresponding abnormal traffic detection model. For example, during the training phase, an abnormal traffic detection model for the D-Link camera 930L category is trained based on normal traffic from devices in the D-Link camera 930L category. During the abnormal traffic detection phase, traffic from the D-Link camera 930L category is input into the corresponding abnormal traffic detection model for the D-Link camera 930L category.

[0101] 4. Federated Learning Module

[0102] This module is primarily responsible for combining multiple, differently distributed datasets to jointly train the anomaly detection model during the training phase. When federated learning mode is enabled, it can be combined with other LANs for joint learning without sharing data, improving the detection accuracy of the pre-set anomaly detection model.

[0103] The specific implementation steps of federated learning are as follows:

[0104] 1) Each training participant (which can be multiple local area networks) downloads the latest model of device type K (for example, device type D-Link Camer 930L) from the security service center or obtains a random model parameter;

[0105] 2) Each participant uses local data to train a model for device type k;

[0106] 3) Each party uploads the model parameters of the local device type k to the security service center;

[0107] 4) The security service center aggregates the parameters of each user to update the model parameters of device type k and determines whether the training ends when a certain number of training rounds is reached or the model loss value is less than a set value. If so, the training ends and the model parameters of device type k are returned to each participant. If not, the training returns to step 1.

[0108] After using federated learning, multiple LANs can jointly train the device type k abnormal traffic detection model without sharing data.

[0109] 5. New device abnormal traffic detection module based on transfer learning

[0110] As the number of major device types in this module continues to increase, the abnormal traffic detection model needs to be retrained from scratch, and updates will incur costs. This module aims to reduce the training cost when new device types appear.

[0111] First, the device type is processed into a word vector using word2vec. When a new device is added, the Euclidean distance is used to calculate the similarity between devices and find the device type most similar to the new device. The formula for the Euclidean distance is as follows:

[0112]

[0113] Where d represents the distance, x i represents the i-th dimension of the x-vector, y i Represents the i-th dimension of the y vector.

[0114] Afterwards, the model parameters of similar devices are extracted to initialize the abnormal traffic detection model of the new device, and the normal traffic data of the new device is used to fine-tune the model to obtain the abnormal traffic detection model of the new device, saving training time.

[0115] This module is mainly aimed at new device types appearing in the network and faster training of device models

[0116] The present invention is specifically described below with two examples:

[0117] Example 1: Abnormal traffic detection based on autoencoder of device type

[0118] Take deployment on a LAN gateway as an example:

[0119] Abnormal traffic detection model training phase

[0120] 1) Use tcpdump to capture the ingress and egress traffic of the specified LAN;

[0121] 2) Use existing device type identification technology to mark the device types in the local area network and generate the corresponding relationship between MAC address and device type;

[0122] 3) Extract traffic features;

[0123] 3) Determine the device type based on the traffic characteristics according to the MAC address, and input it into the 5-layer autoencoding abnormal traffic detection model bound to its device type to train the model;

[0124] 4) Define the normal flow loss value boundary for each device type and use this as the threshold to determine whether it is normal flow for the device.

[0125] Abnormal traffic detection model detection stage

[0126] 1) Use tcpdump to capture the ingress and egress traffic of the specified LAN;

[0127] 2) Use existing device type identification technology to identify the device type corresponding to the traffic;

[0128] 3) Extract traffic features;

[0129] 4) Input the traffic features into the 5-layer self-encoding abnormal traffic detection model bound to its device type to calculate the loss value.

[0130] 5) Traffic exceeding the threshold is considered abnormal traffic.

[0131] Example 2: Abnormal traffic detection model aggregation based on federated learning

[0132] Take deployment on a LAN gateway as an example:

[0133] Abnormal traffic detection model training phase

[0134] 1) Use tcpdump to capture the ingress and egress traffic of the specified LAN;

[0135] 2) Use existing device type identification technology to mark the device types in the local area network and generate the corresponding relationship between MAC address and device type;

[0136] 3) Extract traffic features;

[0137] 4) Input the traffic characteristics into the 5-layer autoencoder abnormal traffic detection model bound to the device type according to the device type and train the model for a fixed number of rounds;

[0138] 5) Upload the trained model parameters to the security service center;

[0139] 6) The security service center aggregates the abnormal traffic detection model parameters of the same device type trained on multiple LANs to determine whether the training ends, such as reaching a certain number of training rounds or the model loss value being less than a set value. If so, the training ends and the model parameters for device type k are returned to each participant. If not, the training continues as in step 1.

[0140] 7) Define the normal flow loss value boundary for each device type and use this as the threshold to determine whether the device is normal.

[0141] Abnormal traffic detection model detection stage

[0142] 1) Use tcpdump to capture the ingress and egress traffic of the specified LAN;

[0143] 2) Use existing device type identification technology to identify the device type corresponding to the traffic;

[0144] 3) Extract traffic features;

[0145] 4) Input the traffic features into the 5-layer self-encoding abnormal traffic detection model bound to its device type to calculate the loss value.

[0146] 5) Traffic exceeding the threshold is considered abnormal traffic.

[0147] Experimental data

[0148] Experimental dataset:

[0149] 1) Normal data set, as shown in Table 1:

[0150] Model Category Normal data volume (flow) computer|macos 86489 computer|windows 80917 d-link|camera|930l 6757 phone|android 68258

[0151] Table 12) Abnormal data set, as shown in Table 2:

[0152]

[0153] Table 2

[0154] 1. Experiment 1:

[0155] The results of classification and non-classification are shown in Table 3:

[0156]

[0157] Experimental conclusion:

[0158] Compared with unclassified technology, the abnormal traffic detection technology based on device type classification has significantly improved the accuracy of identifying abnormal data in botnet and penetration categories, by 13% and 8% respectively. The anomaly recognition accuracy for type3-dlink|camera|d930l can reach 100%, and the false positive rate is 0, which is significantly more effective than unclassified technology.

[0159] 2. Experiment 2

[0160] The experimental results of the unsupervised anomaly detection algorithm based on sparse autoencoder on the computer|windows dataset are as follows:

[0161] As shown in Table 4:

[0162]

[0163] Table 4

[0164] Experimental conclusion:

[0165] The sparse autoencoder has a significantly higher accuracy than the baseline in identifying botnet and infiltration anomaly data in the type2-computer|windows category, and is on par with the baseline in scanning and DOS.

[0166] 3. Experiment 3

[0167] The experimental results of the federated autoencoder algorithm on the computer|windows dataset are shown in Table 5:

[0168]

[0169] Table 5

[0170] Experimental conclusion:

[0171] Compared with LAN-only learning, the device anomaly detection model aggregation technology based on federated learning can effectively utilize the knowledge of multiple LAN data sets and improve the accuracy of anomaly data recognition.

[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting abnormal traffic based on device type, comprising the following steps: Build a mapping relationship between device MAC address and device type; Extract the device MAC address in the traffic to be detected, and based on the mapping relationship, select the abnormal traffic detection model of the corresponding device type, wherein the abnormal traffic detection model is obtained by the following steps: The normal traffic training set of the corresponding device type is used as a training sample; Encoding the L1 paradigm parameters used by the sparse autoencoder as chromosomes of the genetic space, using the mean square error during the training of the sparse autoencoder as fitness, and finding the optimal L1 paradigm parameters with the goal of minimizing fitness; wherein, encoding the L1 paradigm parameters used by the sparse autoencoder as chromosomes of the genetic space, using the mean square error during the training of the sparse autoencoder as fitness, and finding the optimal L1 paradigm parameters with the goal of minimizing fitness, includes: Calculate the number of chromosomes in the population The mean squared error during the training of the sparse autoencoder when used as the l1 norm parameter is used as the fitness; where i is the chromosome number, t is the number of breeding rounds, t≤T, and T is the total number of breeding rounds; Calculate each chromosome in this round of reproduction according to the fitness The probability of being a parent of the next generation and on that chromosome As the next generation of parents, based on the chromosome Perform crossover and mutation to generate the next generation of chromosomes Among them, the crossover refers to the exchange of genes at two fixed positions of two paired chromosomes to form two new chromosomes, and the mutation refers to the exchange of genes at two fixed positions of two paired chromosomes to form two new chromosomes. The fixed position of mutates with a probability to form a new chromosome; In the case of t=T, according to the minimum fitness Get the optimal l1 normal form parameters; Regularize the sparse autoencoder based on the optimal l1 normal form parameters to obtain an abnormal traffic detection model; The flow characteristics of the flow to be detected are input into an abnormal flow detection model of a corresponding device type to obtain an abnormal flow detection result.

2. The method according to claim 1, wherein Methods for obtaining the traffic to be tested include: using wirshark or tcpdump to capture the ingress and egress traffic of a specified LAN.

3. The method according to claim 1, wherein The mapping relationship is constructed by the following steps: 1) Extract device identification features of traffic samples of each device type; 2) Based on the device recognition features and using the multi-view wide-depth algorithm, passive recognition results are obtained; 3) Use nmap to actively scan traffic samples of each device type to obtain active identification results; 4) Based on the passive identification results and the active identification results, the mapping relationship between the device MAC address and the device type is obtained.

4. The method according to claim 3, wherein Device identification features include: broadcast packet features and HTTP traffic features; broadcast packet features include: data packet features of the DHCP protocol, MDNS protocol, SSDP protocol, LLMNR protocol, and NBNS protocol; HTTP traffic features include: user-agent field.

5. The method according to claim 1, wherein Loss function for training sparse autoencoders Where n is the dimension of the sparse autoencoder input vector, x p is the p-th dimension value of the input vector, is the p-th dimension value of the output vector, m is the number of neurons in the middle layer of the sparse autoencoder, λ is the optimal paradigm parameter, and w q is the weight of the qth neuron in the middle layer.

6. The method according to claim 1, wherein Traffic features include: data packet size, flow duration, TTL, window, flag statistics and TCP statistics of the same source IP and destination IP within a fixed time; before the traffic features are input into the abnormal traffic detection model of the corresponding device type, the traffic features are preprocessed; the preprocessing method includes: normalizing the traffic features.

7. The method according to claim 1, wherein When the abnormal traffic detection method is applicable to multiple local area networks, the abnormal traffic detection model is trained by a federated learning method, including: Each training participant u obtains the abnormal traffic detection model of device type k from the security service center Or the abnormal traffic detection model parameters of device type k Based on abnormal traffic detection model Or abnormal traffic detection model parameters Use the local normal traffic training set for training to obtain the abnormal traffic detection model Where v is the training round; Abnormal traffic detection model Parameters Upload to the security service center; Security Service Center based on parameters Obtaining anomaly traffic detection models and abnormal traffic detection model parameters When the training round v reaches the set training round or abnormal traffic detection model The loss value is less than the set value, the abnormal flow detection model As a trained device type k abnormal traffic detection model.

8. The method according to claim 1, wherein Perform the following steps to migrate the anomaly traffic detection model to the new device type: 1) Use the word2vec method to process the original device type and the new device type into word vectors respectively; 2) Based on the Euclidean distance between word vectors, obtain the original device type that is most similar to the new device type; 3) Using the training data of the new device type, the abnormal traffic detection model of the most similar original device type is trained to obtain the abnormal traffic detection model of the new device type.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 8.

Images