Anomaly Detection Method and System for Power Internet of Things Based on Hypergraph Fusion

Abstract

The present invention discloses an abnormal detection method and system for the power Internet of Things based on hypergraph fusion, which captures multi-source heterogeneous data in the power Internet of Things and performs preprocessing; uses an encoder-decoder composed of an LSTM and a Transformer encoder to learn the latent feature representation of the multi-source heterogeneous data, and adds adversarial training during the training process; performs clustering analysis on the learned latent feature representation, and constructs a hypergraph according to the analysis results; adopts the HGCN-Transformer method for weighted fusion of multiple hypergraphs, and uses a hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph; respectively uses a Transformer encoder for prediction and a VAE for reconstruction of the output of the hypergraph attention mechanism, calculates the abnormal score from the predicted value and the reconstruction probability, sets a threshold, compares the abnormal score with the threshold to determine whether it is abnormal, and completes the abnormal detection. The present invention can mine the complex and diverse relationships between multi-source heterogeneous data and improve the accuracy of abnormal detection.

Description

Technical Field

[0001] The present invention relates to the technical field of power Internet of Things anomaly detection, and specifically relates to a power Internet of Things anomaly detection method and system based on hypergraph fusion. Background Art

[0002] With the continuous development of the power Internet of Things, users have put forward higher requirements for the reliability and stability of the network. Anomaly detection is one of the important means to ensure the reliable and stable operation of the power Internet of Things. However, with the growth of access terminals, a large amount of multi-source heterogeneous data has also emerged in the power Internet of Things, resulting in slow anomaly detection speed and low accuracy. Effective detection of multi-source heterogeneous data is an important means to ensure the security of the power Internet of Things.

[0003] Multi-source heterogeneous data presents the characteristics that the data comes from different sources and the data formats and types are inconsistent. It is necessary to perform efficient fusion of multi-source heterogeneous data to improve the accuracy and speed of anomaly detection. According to the hierarchical relationship of data abstraction, data fusion can be divided into the following three categories: data-level fusion, feature-level fusion, and decision-level fusion. Among them, data-level fusion refers to directly fusing the collected raw data information and performing data integration and analysis before preprocessing the raw data of various sensor devices. Feature-level fusion belongs to the intermediate-level fusion. First, features are extracted from the raw information of sensor devices, and then the feature information is comprehensively analyzed and processed. Decision-level fusion collects the same target through different types of sensor devices. Each sensor device completes basic processing such as preprocessing, feature extraction, recognition, or judgment locally, and establishes a preliminary conclusion for the monitored target. Among them, feature-level fusion realizes a considerable amount of information compression, which is beneficial to real-time processing. Therefore, it is more suitable for the framework of deep learning methods and can achieve fast and accurate anomaly detection.

[0004] With the development of deep learning, more and more methods have been applied to power Internet of Things anomaly detection. Facing the increasing number of terminal accesses, more complex and specific methods are needed to model the relationships between multiple terminal accesses. Graph neural networks (GNNs) have been successfully used for relationship representation, from recommendation systems to social networks, and then to disease transmission. In graph neural networks, node features, edge features, and structural features are initialized, and then vectors are output by aggregating node and edge information to represent the graph. However, the learned graph is still limited to binary relationships, that is, an edge can only connect two nodes. Obviously, this is not suitable for the multi-way relationships in multi-source heterogeneous data. Therefore, how to capture multi-way correlations in multi-source heterogeneous data to achieve anomaly detection is crucial for judging the operating state of the system.

[0005] In the invention with the patent application number CN202410442022.7, a multi-source heterogeneous data prediction method is proposed, which is mainly trained according to the following steps: 1) Obtain the data streams of each data source; 2) Preprocess the obtained data streams to obtain the preprocessed data streams; 3) Construct the preprocessed data streams into a heterogeneous graph; 4) Construct a multi-source heterogeneous data prediction model and train it; 5) Input the heterogeneous graph into the pre-trained multi-source heterogeneous data prediction model for prediction to obtain the prediction results. However, this solution has the following defects: ① When constructing a heterogeneous graph from multi-source heterogeneous data, although the edges in the heterogeneous graph can represent different types of relationships, they are still binary, that is, they can only connect two nodes. To represent multi-ary relationships, additional indirect representations of nodes and edges are required, which may lead to the complication of the graph structure. ② The Transformer module is used to perform node aggregation and learn node representations, but this method usually relies on the hierarchical attention mechanism to propagate information layer by layer, which may limit the information propagation of distant nodes and ignore the complex interactions between multiple nodes.

[0006] In the invention with the patent application number CN202110999765.0, a student performance prediction method based on a hypergraph neural network is proposed, which is mainly trained according to the following steps: 1) Extract multi-source behavior characteristics from students' multi-source heterogeneous data; 2) Conduct sensitivity analysis on the multi-source behavior characteristics of all students to obtain the influence characteristics of each behavior; 3) Use the influence characteristics to construct a multi-source behavior hypergraph; 4) Predict the students' performance. However, this solution has the following defects: ① This method uses a decision tree classification model to obtain the influence characteristics of the highest single behavior, but this method is usually used for the classification of static data, and it is difficult to process time series data or data with sequential relationships, and it is prone to overfitting, especially in the case of high feature dimensions and limited sample sizes; ② The method of splicing multiple hypergraphs is used to form the final hypergraph, and this method may lose important information during the splicing process and cannot flexibly fuse multiple hypergraphs.

[0007] The disadvantages of the above existing technologies can be summarized as follows: When processing data by constructing a heterogeneous graph structure in multi-source heterogeneous data, the relationships represented by the edges of the traditional graph structure are still binary, and they cannot well represent the multi-ary relationships of multi-source heterogeneous data, and they cannot well process some time series data with high-dimensional features or data with sequential relationships, and are prone to overfitting. Summary of the Invention

[0008] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a power Internet of Things anomaly detection method and system based on hypergraph fusion.

[0009] To achieve the above object, the present invention provides the following technical solution: A method for anomaly detection in a power Internet of Things based on hypergraph fusion, the detection method comprising the following steps:

[0010] Capture multi-source heterogeneous data in the power Internet of Things, and preprocess the obtained multi-source heterogeneous data;

[0011] Input the preprocessed multi-source heterogeneous data into an encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the latent feature representations of the multi-source heterogeneous data, and add adversarial training during the training process of the encoder-decoder;

[0012] Perform clustering analysis on the learned latent feature representations, and construct a hypergraph according to the analysis results;

[0013] Extract the local structure information of each hypergraph through HGCN, perform weighted fusion on multiple hypergraphs through a Transformer encoder, and use a hypergraph attention mechanism to learn hypergraph feature representations for the fused hypergraph;

[0014] Use a Transformer encoder to implement prediction for the output of the hypergraph attention mechanism, use a variational autoencoder VAE to implement reconstruction, calculate an anomaly score from the predicted value and the reconstruction probability, set a threshold, compare the anomaly score with the threshold, and determine whether it is abnormal to complete anomaly detection.

[0015] In a preferred embodiment, capturing multi-source heterogeneous data in the power Internet of Things and preprocessing the obtained multi-source heterogeneous data includes the following steps:

[0016] Collect data streams from different data sources, including power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, divide the test set and the training set, and perform preprocessing on the collected data, including data cleaning, data standardization, and data format conversion. Among them, data standardization and data format conversion are applicable to both the training set and the test set, while data cleaning is only applicable to the training set;

[0017] Data cleaning: Different methods are used to handle missing values and duplicate values for different types of data streams. Among them, power equipment sensor data and environmental data both belong to time series data. The linear interpolation method is used to fill in the missing values, and for the handling of duplicate values, the time stamps are checked for duplicates, and the duplicate time stamp records are deleted. For the missing values of network traffic data, the mean value is used for filling, and for duplicate values, duplicate data packets are deleted. For the missing values of log data, they are directly skipped, and for duplicate values, duplicate log records are deleted. For the missing values of user behavior data, the mode is used for filling, and for duplicate values, duplicate user operation records are deleted.

[0018] Data standardization: The data after data cleaning is standardized using Min - Max normalization. The formula is as follows:

[0019] ,

[0020] Where respectively represent the original feature value, the normalized feature value, the minimum value in the original feature, and the maximum value in the original feature.

[0021] In a preferred embodiment, the pre - processed multi - source heterogeneous data is respectively input into the encoder - decoder composed of LSTM - Transformer encoder and Transformer encoder - LSTM according to different data types to learn the latent feature representations of the multi - source heterogeneous data, including the following steps:

[0022] According to the different data types of power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, they are input into two encoder - decoders. Among them, power equipment sensor data, network traffic data, and environmental data are input into the encoder - decoder composed of LSTM - Transformer encoder, and log data and user behavior data are input into the encoder - decoder composed of Transformer encoder - LSTM to respectively learn the latent feature representations.

[0023] In a preferred embodiment, adversarial training is added during the training process of the encoder - decoder, including the following steps:

[0024] A discriminator is added during the training process of the encoder - decoder for adversarial training. The discriminator uses a convolutional neural network CNN. The generator during the adversarial training process is the encoder - decoder composed of LSTM and Transformer encoder. During the training process, there are the loss function of the discriminator and the adversarial loss function of the generator. The loss function of the discriminator is as follows:

[0025] ,

[0026] wherein is the real data, is the probability distribution of the real data, is the expectation, is the fake data generated by the generator, and the fake data is generated by the random noise generated, is the probability distribution of the random noise, is the output of the discriminator for the real data i.e., the judgment is the probability that it is real data, is the output of the discriminator for the fake data generated by the generator i.e., the judgment of the probability that the generated data comes from real data, represents the loss of the discriminator on the real data, and the discriminator hopes to maximize , represents the loss of the discriminator on the generated data, and the discriminator hopes to maximize ;

[0027] The goal of the generator is to generate data that can deceive the discriminator. The adversarial loss function of the generator is as follows:

[0028] ,

[0029] wherein the generator hopes to maximize i.e., the generated fake data is judged as real data by the discriminator;

[0030] The total loss function is as follows:

[0031] ,

[0032] wherein represents the weight parameter.

[0033] In a preferred embodiment, clustering analysis is performed on the learned latent feature representation, and a hypergraph is constructed according to the analysis result, including the following steps:

[0034] The hypergraph is constructed by clustering the latent feature representation of the encoder-decoder. Mini-Batch K-Means is used to cluster the latent feature representation, and the formula is described as follows:

[0035] ,

[0036] wherein represents a set of clustering clusters, K represents the number of clusters, represents the sample set of the k-th cluster in the t-th iteration, It belongs to the potential feature representation of the current batch of data. It represents the centroid of the k-th cluster in the t-th iteration; after clustering, it is expressed as:

[0037] ,

[0038] H k is the set of potential feature representations belonging to the k-th cluster. It represents the k-th cluster;

[0039] The hypergraph is defined as , where represents the vertex, corresponding to the potential feature representation of the data. represents the hyperedge. Each hyperedge connects multiple vertices, representing the multi-source relationship between them. According to the clustering results, a hypergraph is constructed, and the centroid of each cluster is used as the vertex of the hypergraph, that is . If the Euclidean distance between the centroids of two clusters is less than the set threshold, a hyperedge is constructed between them.

[0040] In a preferred embodiment, the local structure information of each hypergraph is extracted by HGCN, and multiple hypergraphs are weighted and fused by a Transformer encoder, including the following steps:

[0041] In HGCN, the hypergraph convolution operation is based on the hypergraph Laplacian matrix. The expression of the hypergraph Laplacian matrix is:

[0042] ,

[0043] where I represents the identity matrix. represents the vertex degree matrix. represents the hyperedge degree matrix, that is, the degree of each hyperedge. represents the weight matrix of the hyperedges of the a-th hypergraph. represents the transpose of the weight matrix of the hyperedges of the a-th hypergraph. represents the adjacency matrix of the a-th hypergraph. represents the transpose of the adjacency matrix of the a-th hypergraph. is a matrix of dimension p, where p is the number of vertices. is the number of hyperedges in the a-th hypergraph;

[0044] The hypergraph convolution operation is performed on a single hypergraph by HGCN to aggregate the node features, obtaining the new features after convolution, and the formed new features are input into the Transformer encoder to learn the weights of each hypergraph, and the multiple hypergraphs are weighted and fused by the generated weights. The formula is as follows:

[0045] ,

[0046] where represents the value matrix corresponding to the fused hypergraph features, represents the number of hypergraphs, represents the weight of the a-th hypergraph, represents the value matrix corresponding to the features of the a-th hypergraph.

[0047] In a preferred embodiment, a hypergraph attention mechanism is used to learn the hypergraph feature representation of the fused hypergraph, including the following steps:

[0048] Node-level attention and edge-level attention are respectively introduced to learn the hypergraph feature representation of the fused hypergraph;

[0049] Among them, node-level attention learns the importance of each node in the heterogeneous graph based on the neighbors of the hyperedge and aggregates the representations of the neighbors to form node embeddings; the formula for node-level attention is as follows:

[0050] ,

[0051] where represents the attention weight between node r and node q on the hyperedge , respectively represent the feature vectors of node r and node q, represents the feature vector of the hyperedge , W represents a linear transformation matrix used to linearly transform the features of nodes and hyperedges, represents the transpose of the vector used to calculate the attention weight, represents the LeakyReLU activation function, || represents the concatenation operation of vectors, represents the set of all nodes related to the hyperedge , represents the feature vector of node in the set of all nodes related to the hyperedge , and then the node features are updated according to the correlation coefficient, and the formula is as follows:

[0052] ,

[0053] where represents the aggregated feature representation of node r based on the neighbor nodes of node r on the hyperedge ; finally, node embeddings are formed for different types of hyperedges as , represents the hyperedge type;

[0054] The calculation formula for edge-level attention is as follows:

[0055] ,

[0056] where represents the normalization of the softmax function, represents the hyperedge type is the edge-level attention weight, represents the feature representation of node r under the hyperedge type and represents the transpose of the vector used to calculate the edge-level attention, represents the LeakyReLU activation function, represents the total number of nodes in the graph, which is used to average the features of the nodes, represents the set of all hyperedge types;

[0057] The final hypergraph feature representation , and the formula is as follows:

[0058] ,

[0059] where represents the total number of hyperedge types.

[0060] In a preferred embodiment, the output of the hypergraph attention mechanism is used to implement prediction using a Transformer encoder and reconstruction using a variational autoencoder (VAE). The prediction value and the reconstruction probability are used to calculate the anomaly score, and a threshold is set. The anomaly score is compared with the threshold to determine whether it is an anomaly, including the following steps:

[0061] Using a Transformer encoder as the prediction model, taking the final hypergraph feature representation as the input of the Transformer encoder, and using the root mean square error as the loss function:

[0062] ,

[0063] where represent the true value and the predicted value respectively, represents the timestamp;

[0064] Using a variational autoencoder (VAE) as the reconstruction model, by treating the value of the hypergraph feature representation as a variable, capturing the data distribution of the entire data stream;

[0065] The final loss is: , where represents the reconstruction loss;

[0066] The anomaly score of the timestamp is calculated by the following formula:

[0067] ,

[0068] where is the reconstruction probability, is the predicted value, is a learnable hyperparameter; the threshold is automatically selected using the peak over-threshold method, and timestamps with anomaly scores greater than the threshold are regarded as anomalies.

[0069] An anomaly detection system for the power Internet of Things based on hypergraph fusion, including a preprocessing module, a latent feature extraction module, a hypergraph construction module, a weighted fusion module, a hypergraph feature representation learning module, and an anomaly detection module;

[0070] Preprocessing module: Capture multi-source heterogeneous data in the power Internet of Things and preprocess the obtained multi-source heterogeneous data;

[0071] Latent feature extraction module: Input the preprocessed multi-source heterogeneous data into an encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the latent feature representations of the multi-source heterogeneous data, and add adversarial training during the training process of the encoder-decoder;

[0072] Hypergraph construction module: Perform clustering analysis on the learned latent feature representations and construct a hypergraph according to the analysis results;

[0073] Weighted fusion module: Extract the local structure information of each hypergraph through HGCN and perform weighted fusion on multiple hypergraphs through a Transformer encoder;

[0074] Hypergraph feature representation learning module: Use the hypergraph attention mechanism to learn hypergraph feature representations for the fused hypergraph;

[0075] Anomaly detection module: Use a Transformer encoder to implement prediction for the output of the hypergraph attention mechanism, use a variational autoencoder VAE to implement reconstruction, calculate the anomaly score from the predicted value and the reconstruction probability, set a threshold, compare the anomaly score with the threshold, and determine whether it is an anomaly to complete anomaly detection.

[0076] In the above technical solution, the technical effects and advantages provided by the present invention:

[0077] 1. The present invention introduces a hypergraph neural network into the task of abnormal detection of the power Internet of Things. The hypergraph can model multivariate relationships more accurately. By constructing a hypergraph, the multivariate relationships of multi-source heterogeneous data are represented. Unlike the ordinary graph structure, the hyperedge of the hypergraph can connect multiple vertices. Such a hypergraph structure can better model multi-source heterogeneous data to dig out the relationship between multi-source heterogeneous data, so that multi-source heterogeneous data can be processed more flexibly. Therefore, the present invention adopts the construction of an LSTM-Transformer model as an encoder-decoder for feature extraction of multi-source heterogeneous data. The representation of low-dimensional features can be better learned by reconstruction, and the strategy of adversarial training is introduced in the reconstruction stage, so that low-dimensional features can be better learned. The clustering results represented by low-dimensional features are used to construct a hypergraph of multi-source heterogeneous data to dig out the complex and diverse relationships between multi-source heterogeneous data.

[0078] 2. The present invention adopts the HGCN-Transformer method to perform weighted fusion on the learned multiple hypergraph structures. HGCN-Transformer can dynamically adjust the weights of each hypergraph according to the network status and data, so as to better adapt to environmental changes and provide real-time analysis results. It also uses the hypergraph attention mechanism to learn the representation of hypergraph structure nodes, helping to improve the accuracy of anomaly detection.

[0079] 3. The present invention adopts joint training, which takes into account both the periodic information of the time series and the global data distribution of the time series when judging abnormal sequences. Joint training can combine the advantages of the prediction model and the reconstruction model, while perfectly complementing their shortcomings. In addition, Transformer is introduced into the prediction model, and Transformer can effectively capture the information of the global context, helping to improve the accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0081] Figure 1 is a flow chart of the method of the present invention;

[0082] Figure 2 The figure is a diagram showing the overall structure of the method of the present invention. DETAILED DESCRIPTION

[0083] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0084] Embodiment 1:

[0085] The scale of the power Internet of Things and the complex technical infrastructure with a higher level of automation, connectivity, and remote access make it necessary to use the best and intelligent methods to detect various anomalies. In addition, the multi-source heterogeneous data in the power Internet of Things is complex and diverse, and a large amount of complex data will be generated from different sources and different types of devices. Therefore, in the power Internet of Things, how to effectively process multi-source heterogeneous data has become the main challenge. In response to this, this embodiment provides a method for detecting anomalies in the power Internet of Things based on hypergraph fusion to improve the detection accuracy and speed and reduce the risk of missed reports and false alarms. Different from ordinary graph structures, the hyperedges of a hypergraph can connect multiple vertices. Such a hypergraph structure can better model multi-source heterogeneous data to discover the relationships between multi-source heterogeneous data, so that multi-source heterogeneous data can be processed more flexibly. Therefore, this embodiment uses a constructed LSTM-Transformer model as the encoder-decoder for multi-source heterogeneous data feature extraction. Through the reconstruction method, the representation of low-dimensional features can be better learned, and the strategy of adversarial training is introduced in the reconstruction stage to better learn the low-dimensional features. The hypergraph of multi-source heterogeneous data is constructed based on the clustering results of the low-dimensional feature representation to discover the complex and diverse relationships between multi-source heterogeneous data.

[0086] If the learned multiple hypergraph structures cannot be well fused, some important relationship features may be lost. Moreover, the data stream in the power Internet of Things is usually dynamically changing, resulting in the dynamic change of the hypergraph topology structure, and the dynamically changing hypergraph structures need to be effectively fused. In addition, it is necessary to aggregate and update the node information of the learned hypergraph structures to better learn the representation of the nodes, thereby improving the accuracy of anomaly detection. Therefore, this embodiment uses the HGCN-Transformer method to perform weighted fusion on the learned multiple hypergraph structures. HGCN-Transformer can dynamically adjust the weights of each hypergraph according to the network state and data, so as to better adapt to environmental changes, provide real-time analysis results, and use the hypergraph attention mechanism to learn the representation of the hypergraph structure nodes to help improve the accuracy of anomaly detection.

[0087] Please refer to Figure 1 andFigure 2 As shown in Figure 2 , in this embodiment, a method for detecting anomalies in a power Internet of Things based on hypergraph fusion includes the following steps:

[0088] (1) Capture multi-source heterogeneous data in the power Internet of Things and preprocess the obtained multi-source heterogeneous data. Among them, data cleaning and data standardization are used to preprocess the data, scale the attributes of the multi-source heterogeneous data to a certain range, and replace the outliers in the data with normal values.

[0089] (2) Input the preprocessed multi-source heterogeneous data into an encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM respectively according to different data types to learn the latent feature representations of the multi-source heterogeneous data. By combining LSTM and Transformer, the model can flexibly process different types of data. LSTM can focus on data with strong temporal characteristics (power equipment sensor data, network traffic data, environmental data), while Transformer can process more types of heterogeneous data (log data, user behavior data). Therefore, through the encoder-decoder, data from different sources can be mapped to the same feature space, facilitating comprehensive analysis and modeling. Therefore, for data with strong temporal characteristics, it is input into the encoder-decoder composed of an LSTM-Transformer encoder, and the other two types of heterogeneous data are input into the encoder-decoder composed of a Transformer encoder-LSTM. In addition, adversarial training is added during the training process of the encoder-decoder to enhance the robustness and generalization ability of the model. Adversarial learning can help the model more effectively extract robust features when facing noise and data inconsistencies.

[0090] (3) Perform clustering analysis on the learned latent feature representations and construct a hypergraph according to the analysis results. Use the feature representations as nodes and construct hyperedges according to the clustering results. The nodes in each cluster form a hyperedge, thereby capturing the complex relationships between multiple samples in the feature representations. The present invention uses Mini-Batch K-Means for clustering analysis of latent feature representations. This method updates the cluster centers by using a small part of the data instead of the entire data set. This significantly reduces the memory requirements, enables it to run in a memory-constrained environment, and speeds up the operation speed, providing reasonable clustering results in a shorter time.

[0091] (4) Extract the local structure information of each hypergraph through HGCN, and perform weighted fusion on multiple hypergraphs through a Transformer encoder. Adopt a hypergraph attention mechanism to learn hypergraph feature representations for the fused hypergraph;

[0092] (5) Joint training is carried out by means of prediction and reconstruction. The output of the hypergraph attention mechanism is used to implement prediction by the Transformer encoder and reconstruction by the variational autoencoder VAE respectively. The prediction value and the reconstruction probability are used to calculate the anomaly score. A threshold is set, and the anomaly score is compared with the threshold to determine whether the multivariate time series is abnormal, thus completing the anomaly detection.

[0093] Capture multi-source heterogeneous data in the power Internet of Things, and preprocess the obtained multi-source heterogeneous data, including:

[0094] Collect data streams from different data sources, including power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, and divide the test set and the training set. Due to the multi-source and heterogeneous nature of the data streams, in order to improve the stability of the model, the collected data is preprocessed by data cleaning, data standardization, and data format conversion. Among them, data standardization and data format conversion are applicable to both the training set and the test set, while data cleaning is only applicable to the training set.

[0095] Data cleaning: Different methods are used to process missing values and duplicate values for different types of data streams. Among them, both power equipment sensor data and environmental data belong to time series data, and there may be problems of missing or duplicate sensor readings. Therefore, the linear interpolation method is used to fill in the missing values, and for the processing of duplicate values, the time stamps are checked for duplication, and the duplicate time stamp records are deleted. For the missing values of network traffic data, the mean value is used to fill in, and for duplicate values, the duplicate data packets are deleted, especially the duplicate rows based on features such as time stamps, source IPs, and destination IPs. For the missing values of log data, they can be directly skipped, and for duplicate values, the duplicate log records are deleted. For the missing values of user behavior data, the mode is used to fill in, and for duplicate values, the duplicate user operation records are deleted.

[0096] Data standardization: The data after data cleaning is standardized, and the data is scaled to a certain range in proportion. The purpose of standardization is to eliminate the differences in data features so that the impacts of different features on the model are roughly the same. In the present invention, Min-Max normalization is adopted, and the formula is as follows:

[0097] ,

[0098] where respectively represent the original feature value, the normalized feature value, the minimum value in the original feature, and the maximum value in the original feature.

[0099] The preprocessed multi-source heterogeneous data are respectively input into the encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the latent feature representations of the multi-source heterogeneous data, including:

[0100] The preprocessed data stream is input into the encoder-decoder composed of LSTM and Transformer to extract latent features. By adding adversarial learning to the encoder-decoder architecture, the model will be trained to not only focus on easily learnable features but also be able to handle potential noise or adversarial perturbations. For multi-source heterogeneous data, this method enables the model to learn more robust latent feature representations, not limited to a specific data source or a specific type of features.

[0101] LSTM is composed of multiple layers stacked together. The core part of LSTM is the forget gate, input gate, and output gate. Among them, the forget gate inputs the information of the previous hidden state and the data of the current time step into the Sigmoid function together. The output value ranges from 0 to 1, representing whether to forget. The closer it is to 0, the more it should be forgotten, and the closer it is to 1, the more it should be remembered. Finally, this value will be multiplied by the previous memory to limit the influence of the previous memory on the subsequent memory. The input gate controls how much of the input information at the current time step will be updated into the cell state. The output gate controls the output of the hidden state at the current time step. The hidden state is the final output of LSTM. The output gate determines which parts of the cell state will affect the hidden state, thus determining the content of the output at the current time step. LSTM is suitable for processing time-series data, especially data with time-dependent relationships, and can effectively capture short-term and long-term dependency patterns in the data. Therefore, it is suitable for processing data sources with strong temporal characteristics. Finally, LSTM will output the hidden state of each source data.

[0102] Transformer consists of an encoder and a decoder. The encoder is responsible for processing the input data and converting it into an internal representation that the model can understand, and the decoder is responsible for generating output data based on this internal representation. Transformer can be used to process more types of heterogeneous data. In the present invention, only the Transformer encoder is used.

[0103] According to the different data types of power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, they are input into two encoder-decoders; among them, the power equipment sensor data, network traffic data, and environmental data are input into the encoder-decoder composed of an LSTM-Transformer encoder, and the log data and user behavior data are input into the encoder-decoder composed of a Transformer encoder-LSTM to learn the latent feature representations respectively.

[0104] During the training process of the encoder-decoder, adversarial training is added, including:

[0105] During the training process of the encoder-decoder, a discriminator is added for adversarial training, so as to obtain better latent feature representations. The discriminator uses a convolutional neural network (CNN), which includes the outputs of a convolutional layer, a ReLU activation layer, a max pooling layer, a batch normalization layer, and a fully connected layer, and uses Sigmoid to activate the output layer. The generator in the process of adversarial training is an encoder-decoder composed of an LSTM and a Transformer encoder. Therefore, there are a loss function of the discriminator and an adversarial loss function of the generator during the training process. The loss function of the discriminator is as follows:

[0106] ,

[0107] where is the real data, is the probability distribution of the real data, is the expectation, is the fake data generated by the generator, and the fake data is generated by random noise is the probability distribution of the random noise, is the output of the discriminator for the real data , that is, the probability of judging as the real data, is the output of the discriminator for the fake data generated by the generator, that is, the probability of judging that the generated data comes from the real data, represents the loss of the discriminator on the real data. The discriminator hopes to maximize , that is, hopes that the real data is judged as the real data (the probability is close to 1). represents the loss of the discriminator on the generated data. The discriminator hopes to maximize , that is, hopes that the generated data is judged as fake data (the probability is close to 0). By minimizing the loss of the discriminator, the discriminator continuously improves its ability to distinguish real data and generated data.

[0108] The goal of the generator is to generate data that can deceive the discriminator. Therefore, its loss function is defined as hoping that the generated data is judged as real data by the discriminator, that is, hoping that the output of the discriminator is close to 1. The adversarial loss function of the generator is as follows:

[0109] ,

[0110] where the generator hopes to maximize , that is, the generated fake data ​Judged as real data by the discriminator (probability close to 1), by minimizing the adversarial loss of the generator, the generator continuously adjusts its parameters to make the generated data more realistic, thus deceiving the discriminator.

[0111] The total loss function is as follows:

[0112] ,

[0113] where represents the weight parameter.

[0114] Perform clustering analysis on the learned latent feature representations and construct a hypergraph according to the analysis results, including:

[0115] By clustering the latent feature representations of the encoder-decoder to construct a hypergraph, in this embodiment, Mini-Batch K-Means is used to cluster the latent feature representations, and the formula is described as follows:

[0116] ,

[0117] where represents a set of clustering clusters, K represents the number of clusters, represents the sample set of the k-th cluster in the t-th iteration (this only includes the samples selected from the Mini-Batch in the t-th iteration), belongs to the latent feature representation of the current batch of data, represents the centroid of the k-th cluster in the t-th iteration; after clustering, it can be expressed as:

[0118] ,

[0119] H k is the set of latent feature representations belonging to the k-th cluster, represents the k-th cluster; the features in each cluster share certain similarities and can be regarded as the local structure of the data.

[0120] A hypergraph is a generalization of a graph and can represent multi-source relationships. The edges in an ordinary graph can only connect two vertices, while the hyperedges in a hypergraph can connect any number of vertices. Therefore, a hypergraph has stronger expressive power when dealing with complex relationships. A hypergraph can be defined as , where represents the vertices, corresponding to the latent feature representations of the data, represents the hyperedges, and each hyperedge connects multiple vertices, representing the multi-source relationships between them. Construct a hypergraph according to the clustering results, and the specific steps are as follows: The centroid of each cluster can be used as the vertex of the hypergraph, that is, If the Euclidean distance between the centroids of two clusters is less than a set threshold, a hyperedge is constructed between them.

[0121] HGCN is used to extract the local structural information of each hypergraph, and a Transformer encoder is used to perform weighted fusion on multiple hypergraphs, including:

[0122] After the hypergraph construction is completed, it is necessary to perform weighted fusion on multiple hypergraphs. By using HGCN to extract the local structural information of each hypergraph and a Transformer encoder to perform weighted fusion on multiple hypergraphs, the relationships of multiple hypergraphs can be fully utilized, thereby improving the representation ability. And weights can be adaptively assigned according to different graph structures to make the fusion of multiple hypergraphs more flexible.

[0123] In HGCN, the hypergraph convolution operation is based on the hypergraph Laplacian matrix, and the expression of the hypergraph Laplacian matrix is:

[0124] ,

[0125] where I represents the identity matrix, represents the vertex degree matrix, represents the hyperedge degree matrix, that is, the degree of each hyperedge, represents the weight matrix of the hyperedges of the a-th hypergraph, represents the transpose of the weight matrix of the hyperedges of the a-th hypergraph, represents the adjacency matrix of the a-th hypergraph, represents the transpose of the adjacency matrix of the a-th hypergraph, is a matrix of dimension, where p is the number of vertices, is the number of hyperedges in the a-th hypergraph.

[0126] HGCN is used to perform hypergraph convolution operations on a single hypergraph to aggregate node feature information, obtaining new features after convolution, and inputting the formed new features into the Transformer encoder to learn the weights of each hypergraph, and performing weighted fusion on multiple hypergraphs through the generated weights. The formula is as follows:

[0127] ,

[0128] where represents the value matrix corresponding to the fused hypergraph features, represents the number of hypergraphs, represents the weight of the a-th hypergraph, represents the value matrix corresponding to the features of the a-th hypergraph.

[0129] The hypergraph attention mechanism is adopted to learn the hypergraph feature representation of the fused hypergraph, including:

[0130] Node-level attention and edge-level attention are respectively introduced to learn the hypergraph feature representation of the fused hypergraph;

[0131] Among them, the node-level attention can learn the importance of the neighbors based on the hyperedges for each node in the heterogeneous graph, and aggregate the representations of these meaningful neighbors to form node embeddings; the formula for node-level attention is as follows:

[0132] ,

[0133] where represents the attention weight between node r and node q on the hyperedge , represent the feature vectors of node r and node q respectively, represents the feature vector of the hyperedge (aggregated based on the connected node features), W represents the linear transformation matrix, which is used to linearly transform the features of nodes and hyperedges, represents the transpose of the vector used to calculate the attention weight, represents the LeakyReLU activation function, || represents the concatenation operation of vectors, represents all the node sets related to the hyperedge , represents the feature vector of node in all the node sets related to the hyperedge . Then, the node features can be updated according to the correlation coefficient, and the formula is as follows:

[0134] ,

[0135] where represents the aggregated feature representation of node r based on the neighbor nodes of node r on the hyperedge ; finally, the node embeddings are formed for different types of hyperedges as , represents the hyperedge type.

[0136] The edge-level attention needs to consider the weight differences between different hyperedge types. Then, the calculation formula for the edge-level attention is as follows:

[0137] ,

[0138] where represents the normalization of the softmax function. The role of the softmax function is to convert a set of input values into a probability distribution through exponential operations and normalization; represents the edge-level attention weight of the hyperedge type , Denote the feature representation of node \(r\) under the hyperedge type and denote the transpose of the vector used to calculate the edge-level attention. Denote the LeakyReLU activation function. The role of the LeakyReLU activation function is to introduce a small non-zero slope in the negative value region to avoid the problem that the ReLU activation function completely fails when the input is negative, thereby alleviating the phenomenon of "neuron death". Denote the total number of nodes in the graph, which is used to average the features of the nodes. Denote the set of all hyperedge types.

[0139] Obtain the final hypergraph feature representation , and the formula is as follows:

[0140] ,

[0141] where denote the total number of hyperedge types.

[0142] Use the output of the hypergraph attention mechanism to implement prediction using the Transformer encoder respectively, use the variational autoencoder VAE to implement reconstruction, calculate the anomaly score for the predicted value and the reconstruction probability, set a threshold, compare the anomaly score with the threshold to determine whether it is abnormal, including the following steps:

[0143] Adopt the Transformer encoder as the prediction model, use the final hypergraph feature representation as the input of the Transformer encoder, and use the root mean square error as the loss function at the same time:

[0144] ,

[0145] where denote the true value and the predicted value respectively, denote the timestamp;

[0146] Adopt the variational autoencoder VAE as the reconstruction model, which provides a probabilistic way to describe the observations in the latent space. By regarding the values of the hypergraph feature representation as variables, the data distribution of the entire data stream can be captured.

[0147] So the final loss is: , where denote the reconstruction loss.

[0148] Calculate the anomaly score of the timestamp through the following formula:

[0149] ,

[0150] where is the reconstruction probability, is the predicted value, is a learnable hyperparameter; in this embodiment, the peak over-threshold method is used to automatically select the threshold, and the timestamps with abnormal scores greater than the threshold are regarded as abnormal.

[0151] Embodiment 2:

[0152] This embodiment provides an abnormal detection system for the power Internet of Things based on hypergraph fusion, including a preprocessing module, a potential feature extraction module, a hypergraph construction module, a weighted fusion module, a hypergraph feature representation learning module, and an abnormal detection module;

[0153] Preprocessing module: Capture multi-source heterogeneous data in the power Internet of Things and preprocess the obtained multi-source heterogeneous data;

[0154] Potential feature extraction module: Input the preprocessed multi-source heterogeneous data into the encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the potential feature representations of the multi-source heterogeneous data, and add adversarial training during the training process of the encoder-decoder;

[0155] Hypergraph construction module: Perform clustering analysis on the learned potential feature representations and construct a hypergraph according to the analysis results;

[0156] Weighted fusion module: Extract the local structure information of each hypergraph through HGCN and perform weighted fusion on multiple hypergraphs through a Transformer encoder;

[0157] Hypergraph feature representation learning module: Use the hypergraph attention mechanism to learn the hypergraph feature representations of the fused hypergraph;

[0158] Abnormal detection module: Use a Transformer encoder to implement prediction on the output of the hypergraph attention mechanism respectively, use a variational autoencoder VAE to implement reconstruction, calculate the abnormal score from the predicted value and the reconstruction probability, set a threshold, compare the abnormal score with the threshold to determine whether it is abnormal, and complete the abnormal detection.

[0159] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0160] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only the specific embodiments. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. An abnormal detection method for the power Internet of Things based on hypergraph fusion, characterized in that: The detection method includes the following steps: Capture multi-source heterogeneous data in the power Internet of Things, and preprocess the obtained multi-source heterogeneous data; Input the preprocessed multi-source heterogeneous data into an encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the latent feature representation of the multi-source heterogeneous data, and add adversarial training during the training process of the encoder-decoder; Perform clustering analysis on the learned latent feature representation, and construct a hypergraph according to the analysis results; Extract the local structure information of each hypergraph through HGCN, perform weighted fusion on multiple hypergraphs through a Transformer encoder, and use a hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph; Use a Transformer encoder to implement prediction for the output of the hypergraph attention mechanism, use a variational autoencoder VAE to implement reconstruction, calculate the anomaly score from the predicted value and the reconstruction probability, set a threshold, compare the anomaly score with the threshold, and determine whether it is abnormal to complete anomaly detection; Perform clustering analysis on the learned latent feature representation, and construct a hypergraph according to the analysis results, including the following steps: Construct a hypergraph by clustering the latent feature representation of the encoder-decoder, and use Mini-BatchK-Means to cluster the latent feature representation. The formula is described as follows: ， Among them represents a set of clustering clusters, and K represents the number of clusters. represents the sample set of the k-th cluster in the t-th iteration. belongs to the potential feature representation of the current batch of data. represents the centroid of the k-th cluster in the t-th iteration; after clustering, it is expressed as: ， H k is a set of potential feature representations belonging to the k-th cluster, representing the k-th cluster; The hypergraph is defined as , where represents vertices, corresponding to the potential feature representation of the data, represents hyperedges, and each hyperedge connects multiple vertices, representing the multi-source relationship between them; the hypergraph is constructed according to the clustering results, and the centroid of each cluster is used as the vertex of the hypergraph, that is . If the Euclidean distance between the centroids of two clusters is less than the set threshold, a hyperedge is constructed between them.

2. The abnormal detection method for the power Internet of Things based on hypergraph fusion according to claim 1, characterized in that: Capture multi-source heterogeneous data in the power Internet of Things, and preprocess the obtained multi-source heterogeneous data, including the following steps: Collect data streams from different data sources, including power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, divide the test set and the training set, and perform preprocessing on the collected data, including data cleaning, data standardization, and data format conversion. Among them, data standardization and data format conversion are applicable to both the training set and the test set, while data cleaning is only applicable to the training set; Data cleaning: Use different methods to process missing values and duplicate values for different types of data streams. Among them, both power equipment sensor data and environmental data belong to time series data. Use linear interpolation to fill in the missing values, and use checking time stamps for duplicates and deleting duplicate time stamp records for processing duplicate values. For missing values in network traffic data, use mean filling, and for duplicate values, use deleting duplicate data packets. For missing values in log data, directly skip them, and for duplicate values, use deleting duplicate log records. For missing values in user behavior data, use mode filling, and for duplicate values, use deleting duplicate user operation records; Data standardization: Perform data standardization on the data after data cleaning, using Min-Max normalization. The formula is as follows: ， wherein respectively represent the original eigenvalue, the normalized eigenvalue, the minimum value in the original features, and the maximum value in the original features.

3. The anomaly detection method for the power Internet of Things based on hypergraph fusion according to claim 2, wherein: Input the preprocessed multi-source heterogeneous data into an encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the latent feature representation of the multi-source heterogeneous data, including the following steps: According to the different data types of power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, they are input into two encoder-decoders. Among them, power equipment sensor data, network traffic data, and environmental data are input into the encoder-decoder composed of LSTM-Transformer encoders, and log data and user behavior data are input into the encoder-decoder composed of Transformer encoder-LSTM to learn latent feature representations respectively.

4. The anomaly detection method for the power Internet of Things based on hypergraph fusion according to claim 3, wherein: During the training process of the encoder-decoder, adversarial training is added, including the following steps: During the training process of the encoder-decoder, a discriminator is added for adversarial training. The discriminator uses a convolutional neural network CNN. The generator during the adversarial training process is an encoder-decoder composed of LSTM and Transformer encoders. There are a loss function of the discriminator and an adversarial loss function of the generator during the training process. The loss function of the discriminator is as follows: ， where is the real data, is the probability distribution of the real data, is the expectation, is the fake data generated by the generator, and the fake data is generated by the random noise ; is the probability distribution of the random noise, is the output of the discriminator for the real data , that is, the probability of judging being the real data, is the output of the discriminator for the fake data generated by the generator , that is, the probability of judging that the generated data comes from the real data, represents the loss of the discriminator on the real data, and the discriminator hopes to maximize , represents the loss of the discriminator on the generated data, and the discriminator hopes to maximize ; The goal of the generator is to generate data that can deceive the discriminator. The adversarial loss function of the generator is as follows: ， where the generator hopes to maximize , that is, the generated fake data is judged as real data by the discriminator; The total loss function is as follows: ， wherein represents a weight parameter.

5. The abnormal detection method for the power Internet of Things based on hypergraph fusion according to claim 4, wherein: Extract the local structure information of each hypergraph through HGCN, and perform weighted fusion on multiple hypergraphs through the Transformer encoder, including the following steps: In HGCN, the hypergraph convolution operation is based on the hypergraph Laplacian matrix, and the expression of the hypergraph Laplacian matrix is: ， where I represents the identity matrix, represents the vertex degree matrix, represents the hyperedge degree matrix, that is, the degree of each hyperedge, represents the weight matrix of the hyperedges of the a-th hypergraph, represents the transpose of the weight matrix of the hyperedges of the a-th hypergraph, represents the adjacency matrix of the a-th hypergraph, represents the transpose of the adjacency matrix of the a-th hypergraph, is a matrix of dimension, where p is the number of vertices, is the number of hyperedges in the a-th hypergraph; Perform hypergraph convolution operation on a single hypergraph through HGCN to aggregate node features to obtain new features after convolution, and input the formed new features into the Transformer encoder to learn the weights of each hypergraph, and perform weighted fusion on multiple hypergraphs through the generated weights. The formula is as follows: ， Among them represents the value matrix corresponding to the fused hypergraph features represents the number of hypergraphs represents the weight of the a-th hypergraph represents the value matrix corresponding to the a-th hypergraph features 6. The abnormal detection method of the power Internet of Things based on hypergraph fusion according to claim 5, characterized in that: Adopt the hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph, including the following steps: Introduce node-level attention and edge-level attention respectively to learn the hypergraph feature representation of the fused hypergraph; Among them, node-level attention learns the importance of each node in the heterogeneous graph based on the neighbors of the hyperedge and aggregates the representations of the neighbors to form node embeddings. The formula for node-level attention is as follows: ， Among them represents the attention weight between node r and node q on the hyperedge , and represent the feature vectors of node r and node q respectively represents the hyperedge . W represents the linear transformation matrix used to perform linear transformation on the features of nodes and hyperedges represents the transpose of the vector used to calculate the attention weight represents the LeakyReLU activation function, and || represents the concatenation operation of vectors represents all node sets related to the hyperedge , and represents the feature vector of node in all node sets related to the hyperedge . Then, the node features are updated according to the correlation coefficient, and the formula is as follows ， Among them represents the feature representation of node r aggregated based on the neighbor nodes of node r on the hyperedge ; finally, for hyperedges of different types, the node embeddings are formed as , represents the hyperedge type; The calculation formula for edge-level attention is as follows: ， Among them represents the normalization of the softmax function represents the hyperedge type is the edge-level attention weight represents the feature representation of node r under the hyperedge type is the feature representation of node r under the hyperedge type represents the transpose of the vector used to calculate the edge-level attention represents the LeakyReLU activation function represents the total number of nodes in the graph, which is used to average the features of the nodes represents the set of all hyperedge types; Final hypergraph feature representation , and the formula is as follows: ， Among them represents the total number of hyperedge types.

7. The abnormal detection method for the power Internet of Things based on hypergraph fusion according to claim 6, characterized in that: Use the Transformer encoder to implement prediction for the output of the hypergraph attention mechanism respectively, use the variational autoencoder VAE to implement reconstruction, calculate the anomaly score from the predicted value and the reconstruction probability, set a threshold, compare the anomaly score with the threshold, and determine whether it is abnormal, including the following steps: Adopt the Transformer encoder as the prediction model, use the final hypergraph feature representation as the input of the Transformer encoder, and use the root mean square error as the loss function at the same time: ， where represent the true value and the predicted value respectively, represents the timestamp; Adopt the variational autoencoder VAE as the reconstruction model, and capture the data distribution of the entire data stream by regarding the value of the hypergraph feature representation as a variable; The final loss is: , where represents the reconstruction loss; Calculate the anomaly score of the timestamp through the following formula: ， where is the reconstruction probability, is the predicted value, is a learnable hyperparameter; the threshold is automatically selected using the peak over threshold method, and timestamps with anomaly scores greater than the threshold are considered anomalies.

8. An abnormal detection system for the power Internet of Things based on hypergraph fusion, which is used to implement the detection method described in any one of claims 1-7, and is characterized in that: Including a preprocessing module, a latent feature extraction module, a hypergraph construction module, a weighted fusion module, a hypergraph feature representation learning module, and an anomaly detection module; Preprocessing module: Capture multi-source heterogeneous data in the power Internet of Things and preprocess the obtained multi-source heterogeneous data; Potential feature extraction module: Input the preprocessed multi-source heterogeneous data into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, and add adversarial training during the training process of the encoder-decoder; Hypergraph construction module: Perform clustering analysis on the learned potential feature representation and construct a hypergraph according to the analysis results; Weighted fusion module: Extract the local structure information of each hypergraph through HGCN, and perform weighted fusion on multiple hypergraphs through the Transformer encoder; Hypergraph feature representation learning module: Use the hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph; Anomaly detection module: Implement prediction for the output of the hypergraph attention mechanism using the Transformer encoder respectively, and implement reconstruction using the variational autoencoder VAE. Calculate the anomaly score from the predicted value and the reconstruction probability, set a threshold, compare the anomaly score with the threshold, and determine whether it is abnormal to complete anomaly detection.

Images