A distributed internet of things device anomaly detection method

Abstract

The application relates to a distributed Internet of Things equipment anomaly detection method and belongs to the Internet of Things field. The method is characterized in that: firstly, the potential feature distribution of abnormal data is learned, feature correction and feature discrimination are performed, high-quality reconstructed data are obtained, the proportion of normal data and abnormal data is balanced, the training of an anomaly detection network is guided through the balanced data, a discrimination threshold is output, and abnormal data is identified. Then, a federated learning algorithm based on dynamic model selection is adopted to calculate the confidence of each model and dynamically select a local model to upload to a central server for model aggregation. The application can realize high-precision anomaly detection of high-dimensional and unbalanced data in an Internet of Things scene and guarantee the generalization ability of a model.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of Internet of Things, and relates to a distributed Internet of Things device anomaly detection method. BACKGROUND

[0002] The Internet of Things refers to connecting any object with a network according to an agreed protocol through information sensing equipment, and performing information interaction and communication through an information transmission medium. The purpose of the Internet of Things is to realize long-term and rapid connection without being limited by place and time, such as mobile devices in application environments such as smart home, smart city and smart traffic. The advent of the 5G era provides numerous technical supports for the development of the Internet of Things, and the formulation of the 5G standard can well meet the needs of the Internet of Things, such as network speed, capacity, security, etc., promote the development of unmanned driving technology, help the development of industrial Internet of Things, and accelerate the development of the virtual reality field of the Internet of Things.

[0003] With the continuous development of the Internet of Things, a large number of devices and users lead to a dramatic increase in mobile data demand, and put forward requirements such as high capacity, ultra-low delay, and better service quality and user experience provided to end users. Complex technologies such as beamforming, large-scale MIMO and dense microcells are born, but these technologies increase the complexity of the system architecture, making network anomaly detection too complex, especially for handling intermittent hardware failures and configuration errors in large networks, which brings the following challenges to anomaly detection: 1) a large amount of unlabeled data war; 2) data imbalance; 3) high-dimensional data heterogeneity.

[0004] The existing device anomaly detection method regards the original data as normal data, ignores the randomness of the anomaly, cannot evaluate the influence degree of the abnormal data on the model training, and cannot meet the difference of the model for detecting heterogeneous devices. The traditional anomaly detection has low detection rate for unbalanced data and poor model generalization ability, and therefore, a new Internet of Things device anomaly detection method is urgently needed. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a distributed Internet of Things device anomaly detection method, which realizes anomaly detection of Internet of Things devices by establishing a new detection network, and improves the robustness of the model for abnormal device data and the extraction accuracy of the feature extractor.

[0006] To achieve the above purpose, the present application provides the following technical scheme:

[0007] A distributed Internet of Things device anomaly detection method, specifically comprising the following steps:

[0008] S1: Obtain high-dimensional time series data of Internet of Things devices and pre-process the data; and the pre-processed abnormal data is used as a training set;

[0009] S2: Feature extraction, feature correction and abnormal feature discrimination are performed on the training set to obtain high-quality reconstructed abnormal data;

[0010] S3: The training set after data expansion is mixed with the original data in equal amount as a validation set to guide the training of the anomaly detection network. The relative entropy of the latent feature distribution of the original data and the latent feature distribution of the reconstructed data is calculated to determine the deviation degree of the reconstructed data from the original data. The higher the deviation degree, the more abnormal it is.

[0011] S4: A federated learning algorithm based on dynamic model selection is used. The validation set is used to initialize the global model, which is then distributed to each participant. After receiving the model, the participants start model training based on their locally collected time series data, and selectively upload the model to the central server for model aggregation.

[0012] Further, in step S1, the high-dimensional time series historical data of the Internet of Things device includes time information, device type, device parameter, device location, and multi-dimensional and multi-scene feature data for identification. The high-dimensional time series data is preprocessed, which specifically includes the following steps:

[0013] S11: Data normalization processing: the data is normalized by using a standardization formula;

[0014] S12: The original data is dimensionally reduced and clustered using the Kmeans clustering algorithm and PCA dimensionality reduction processing;

[0015] S13: Two sliding window processing: the first sliding window increases the correlation between dimensions, and the second sliding window increases the time correlation.

[0016] Further, in step S3, the anomaly detection network is an improved auto-encoding network that introduces an adversarial mechanism in the latent feature layer, i.e., adding a discriminator in the latent feature layer and adding consistency enhancement constraints and shrinkage constraints to ensure that the network produces reasonable and consistent latent representations for abnormal data and maps all input data to similar positions, improving the extraction accuracy of the feature extractor.

[0017] Further, in step S3, the encoder-decoder part of the improved auto-encoding network is composed of two functional networks, which learn the latent feature distribution of the data and increase the diversity of the generated samples by sampling noise.

[0018] Further, in step S3, the anomaly detection network is trained, which specifically includes the following steps:

[0019] S31: Add an encoder layer after the improved auto-encoding network to extract the latent feature distribution of the reconstructed data;

[0020] S32: Calculate the latent feature distribution of the input data by reconstructing the data and the encoding layer;

[0021] S33: Calculate the latent feature distribution of the input data by reconstructing the data and the encoding layer;

[0022] S34: Calculate the relative entropy of the two feature distributions to determine the deviation degree of the reconstructed data from the original data, and record it as an anomaly score, and determine whether the data is abnormal according to the detection threshold.

[0023] Further, step S4 specifically comprises the following steps:

[0024] S41: The anomaly detection model is used as a global model, and the parameters are broadcast to the entire network in the form of blocks, and the terminal device trains a local model based on local data; wherein each local anomaly detection model comprises a local model and an anomaly detection network;

[0025] S42: Calculate the average detection error of each local anomaly detection model, and score the local model according to the average detection error, and calculate the confidence of each model; if the detection error of the local model is higher than that of the global model, the score value of the model is 0, otherwise, the gradient of the model is uploaded according to the confidence;

[0026] S43: The server side aggregates each local anomaly detection model and uses it for the next round of iteration until the model converges, completes training, and returns the global optimal model.

[0027] The beneficial effects of the present application are:

[0028] (1) The anomaly detection framework based on the adversarial double encoding network is adopted, which solves the problem that the detection accuracy of the data imbalance anomaly detection network cannot be guaranteed when the data lacks labels, and improves the robustness of the model to abnormal device data. The present application more effectively highlights the anomaly and suppresses the influence of positive samples on the later model training, improves the generation training of the data in the early stage, and improves the extraction accuracy of the feature extractor.

[0029] (2) The present application also proposes a federated learning method based on model selection, which can dynamically select model uploading and suppress unsatisfactory models, improve the model generalization ability, optimize the model aggregation speed, and to a certain extent, compress the content uploaded to the server, reduce the communication time and energy consumption.

[0030] Other advantages, objects, and features of the present application will be set forth in part in the following specification taken in conjunction with the accompanying drawings, and in part will become apparent to those skilled in the art from a consideration of the following specification and from the practice of the present application. The objects and other advantages of the present application can be realized and attained by means of the instrumentalities and combinations pointed out in the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0032] Figure 1 This is the flow chart of the raw data preprocessing process;

[0033] Figure 2 To combat the overall architecture of the dual encoder network;

[0034] Figure 3 This is a structural diagram of the abnormal data generator;

[0035] Figure 4 Schematic diagram of the structure of abnormal data discriminator;

[0036] Figure 5 This is a schematic diagram of the overall abnormality detection process of the present invention;

[0037] Figure 6 A diagram of the process selected based on the user model. DETAILED DESCRIPTION

[0038] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0039] See also Figures 1-6 , Figure 5 The distributed IoT device anomaly detection method proposed by the present invention mainly includes the following contents:

[0040] 1) Using an adversarial dual encoder network, we actively learn the latent feature distribution of the data, sample the latent features, reconstruct the data, and expand the abnormal data. This balances the ratio of positive and negative samples and improves the model's robustness to abnormal data detection.

[0041] 2) An adversarial mechanism is introduced in the latent feature layer, a discriminator is added to the latent feature layer, and consistency enhancement constraints and contraction constraints are added to ensure that the network produces a reasonable and consistent latent representation for abnormal data and maps all input data to similar locations, thereby improving the extraction accuracy of the feature extractor.

[0042] 3) Calculate the corresponding confidence score for each local model, dynamically select part of the model to participate, accelerate the aggregation of the model, and the model parameters do not need to load a large amount of original data transmission, save the communication bandwidth between the central controller and the local gateway, and to a certain extent, also protect the user privacy.

[0043] The anomaly detection method specifically comprises the following steps:

[0044] S1: Obtain the original data of the high-dimensional time series of the Internet of Things device, and pre-process the original data. The pre-processed historical abnormal data is used as a training set.

[0045] The original data processing step comprises:

[0046] S11: Data normalization processing: the data is normalized by using a standardization formula;

[0047] S12: Dimensionality reduction clustering is performed on the original data using Kmeans+PCA;

[0048] S13: Twice sliding window processing: the first sliding window increases the correlation between dimensions, and the second sliding window increases the time correlation.

[0049] S2: Feature extraction, feature correction and feature discrimination are performed on the training set to expand the abnormal data.

[0050] In this step, the auto-encoding network is improved, wherein the encoder part is composed of two functional networks, the latent feature distribution of the abnormal data is learned, and the sampling noise increases the diversity of the generated samples.

[0051] Then, an adversarial training is introduced in the latent feature layer to ensure that the network can produce reasonable and consistent latent representations for abnormal data and map all input data to similar positions, and a shrinkage constraint of the latent feature space is added to punish sample points with large deviation from the distribution of most samples in the latent feature space.

[0052] S3: After mixing the expanded training set and the original data in equal amounts as a validation set, an anomaly detection algorithm based on a local detection threshold is used to calculate the distribution of the original data after passing through the encoder and the reconstructed data after passing through the decoder, respectively, and the relative entropy of the two distributions is calculated to determine the deviation degree of the reconstructed data from the original data, which is recorded as an anomaly score, to determine the anomaly detection threshold.

[0053] S4: A federated learning method based on model selection is used, the validation set is used to initialize a global model, the model is distributed to each participant, after receiving the model, the participant starts model training according to the time series data collected locally, and selectively selects the model to upload to the central server for model aggregation.

[0054] Embodiment 1:

[0055] This embodiment first carries out data enhancement on abnormal data, balances positive and negative sample data, and then carries out abnormal detection.

[0056] Among them, the data is preprocessed, and the flow chart is as shown in Figure 1

[0057] This embodiment adopts an adversarial double-encoding network, and the overall architecture of the model includes three functions, namely data generation, feature correction and abnormal feature discrimination, and the overall network architecture diagram is as shown in Figure 2

[0058] Next, they will be introduced one by one:

[0059] 1) Data generation:

[0060] In order to solve the problem of single sample generated by traditional autoencoder, the encoder of traditional autoencoder network is improved in the application, and the encoder is composed of two functional networks, and the latent features of the data are learned respectively. The latent features are generated by sampling, and the sampling noise increases the diversity of the generated samples. The overall network architecture diagram is as shown in Figure 3

[0061] 2) Feature correction:

[0062] During data preprocessing, there may be a situation that the positive sample is misjudged as abnormal data, therefore, adversarial training is carried out in the latent feature layer, to ensure that the network can produce reasonable and consistent latent representation for abnormal data and map all input data to similar positions, minimize the influence of normal samples and sampling noise on the reconstruction of abnormal data. And add feature shrinkage constraint, punish sample points with large distribution deviation from most sample points in the latent feature space.

[0063] 3) Abnormal feature discrimination:

[0064] The balanced data is input into the abnormal detection model, and the model is trained. The relative entropy of the latent feature distribution of the original data and the reconstructed data is calculated to represent the deviation degree of the original data and the reconstructed data, and a more reasonable decision threshold is output. By comparing the value of cross entropy and the decision threshold, it is determined whether the data is abnormal. The overall network architecture diagram is as shown in Figure 4

[0065] ​​​​The anti-dual coding network adopted by the present application can solve the problem of anomaly detection caused by large data volume and data imbalance to a certain extent, accelerate detection and improve detection accuracy. However, the premise of machine learning for fault detection is that the data processed is independent and identically distributed, but the distribution of actual data is inconsistent under different frequency of use, which leads to that when the data is not in the same feature space or the data does not satisfy the independent and identically distributed, the machine learning will reconstruct the model according to the new data set, or cause the model to present differentiation in detection effect on different data sets. Secondly, data privacy has become a growing concern for many users. Federated learning solves this problem, which allows multiple participants to jointly train a deep learning model according to their combined data without any participant showing their data to a centralized server. However, this collaborative learning for privacy protection is at the cost of a large amount of communication overhead during training. And once the number of clients exceeds a certain threshold, it may not be possible to communicate weight updates directly, because the workload of communication and aggregation updates grows linearly with the number of clients. Therefore, the present application improves the traditional federated learning:

[0066] 1) Compress the content uploaded to the server to reduce communication time and energy consumption.

[0067] 2) Partial participation, dynamically suppress unsatisfactory models, improve model generalization ability, and optimize model aggregation speed.

[0068] Finally, it should be pointed out that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the present technical solutions, which should be covered in the scope of the claims of the present application.

Claims

1. A distributed method for detecting anomalies in Internet of Things devices, the method comprising: The method specifically comprises the following steps: S1: Obtain high-dimensional time series data of the Internet of Things device and pre-process the same; take the pre-processed abnormal data as a training set; pre-process the high-dimensional time series data, specifically comprising the following steps: S11: data normalization processing: normalize the data by using a standardization formula; S12: use a Kmeans clustering algorithm and PCA dimension reduction processing on the original data to perform dimension reduction clustering; S13: twice sliding window processing; S2: perform feature extraction, feature correction and abnormal feature discrimination on the training set abnormal data to obtain reconstructed abnormal data; S3: mix the training set after data expansion with the original data in equal amounts as a verification set to guide the training of the abnormal detection network; calculate the relative entropy of the latent feature distribution of the original data and the latent feature distribution of the reconstructed data to determine the deviation degree of the reconstructed data from the original data; The abnormal detection network is an improved auto-encoding network, which introduces an adversarial mechanism in the latent feature layer, that is, adds a discriminator in the latent feature layer and adds consistency enhancement constraints and shrinkage constraints to ensure that the network produces reasonable and consistent latent representations for abnormal data and maps all input data to similar positions; the encoder and decoder part of the improved auto-encoding network is composed of two functional networks for calculating the latent feature distribution of the data; S4: use a federated learning algorithm based on dynamic model selection to initialize a global model using the verification set, distribute the model to each participant, and after receiving the model, the participant starts model training based on the time series data collected locally, and selectively uploads the model to the central server for model aggregation.

2. The distributed IoT device anomaly detection method of claim 1, wherein, In step S3, the abnormal detection network is trained, specifically comprising the following steps: S31: add an encoder layer after the improved auto-encoding network to extract the latent feature distribution of the reconstructed data; S32: pass the verification set through the reconstructed data and the encoding layer to calculate the latent feature distribution of the input data; S33: pass the reconstructed data through the newly added decoding layer to calculate the latent feature distribution of the input data; S34: calculate the relative entropy of the two feature distributions to determine the deviation degree of the reconstructed data from the original data, recorded as an abnormal score, and determine whether the data is abnormal according to the detection threshold. 3.The distributed IoT device anomaly detection method of claim 1, wherein, Step S4 specifically comprises the following steps: S41: take the abnormal detection model as a global model, broadcast its parameters to the entire network in blocks, and the terminal device trains a local model based on local data; wherein each local abnormal detection model comprises a local model and an abnormal detection network; S42: calculate the average detection error of each local abnormal detection model, score the local model according to the average detection error, and calculate the confidence of each model; if the detection error of the local model is higher than that of the global model, set the score value of the model to 0, otherwise upload the model with compressed gradient according to the confidence; S43: the server side aggregates each local abnormal detection model and uses it for the next iteration until the model converges, completes the training, and returns the global optimal model.

Images