Power Internet of Things anomaly detection method and system based on hypergraph fusion

Abstract

The invention discloses a hypergraph fusion-based power Internet of Things anomaly detection method and system. The method comprises the steps of capturing and preprocessing multi-source heterogeneous data in a power Internet of Things; learning potential feature representation of the multi-source heterogeneous data by using an encoder-decoder consisting of an LSTM (Long Short Term Memory) and a Transform encoder, and adding adversarial training in a training process; performing clustering analysis on the learned potential feature representation, and constructing a hypergraph according to an analysis result; carrying out weighted fusion on a plurality of hypergraphs by adopting an HGCN-Transform mode, and learning hypergraph feature representation on the fused hypergraphs by adopting a hypergraph attention mechanism; the output of the hypergraph attention mechanism is predicted through a Transform encoder, reconstruction is achieved through VAE, the abnormal score is calculated through a predicted value and the reconstruction probability, a threshold value is set, the abnormal score is compared with the threshold value, whether the abnormal score is abnormal or not is judged, and anomaly detection is completed. According to the method, complex and diversified relationships among multi-source heterogeneous data can be mined, and the anomaly detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of anomaly detection in electric power Internet of Things, and in particular to an anomaly detection method and system for electric power Internet of Things based on hypergraph fusion. Background Art

[0002] With the continuous development of the power Internet of Things, users have put forward higher requirements for the reliability and stability of the network. Anomaly detection is one of the important means to ensure the reliable and stable operation of the power Internet of Things. However, with the increase in access terminals, a large amount of multi-source heterogeneous data has also appeared in the power Internet of Things, resulting in slow anomaly detection speed and low accuracy. Effective detection of multi-source heterogeneous data is an important means to ensure the security of the power Internet of Things.

[0003] Multi-source heterogeneous data presents the characteristics of data coming from different sources and inconsistent data formats and types. It is necessary to efficiently fuse multi-source heterogeneous data to improve the accuracy and speed of anomaly detection. According to the hierarchical relationship of data abstraction, data fusion can be divided into the following three categories: data layer fusion, feature layer fusion, and decision fusion. Among them, data layer fusion refers to the direct fusion of the collected raw data information, and the data integration and analysis are performed before the raw data of various sensor devices are preprocessed. Feature layer fusion belongs to the intermediate level of fusion. First, features are extracted from the raw information of sensor devices, and then the feature information is comprehensively analyzed and processed. Decision fusion collects the same target through different types of sensor devices. Each sensor device completes basic processing such as preprocessing, feature extraction, recognition or judgment locally, and establishes a preliminary conclusion on the monitored target. Among them, feature layer fusion achieves considerable information compression, which is conducive to real-time processing. Therefore, it is more suitable for the framework of deep learning methods and can achieve fast and accurate anomaly detection.

[0004] With the development of deep learning, more and more methods are applied in anomaly detection of power Internet of Things. Faced with more and more terminal accesses, more complex and specific methods are needed to model the relationship between multiple terminal accesses. Graph neural network (GNN) has been successfully used for relationship representation, from recommendation systems to social networks to disease propagation. In graph neural network, node features, edge features and structural features are initialized, and then vector representation of graph is output by aggregating node and edge information. However, the learned graph is still limited to binary relations, that is, an edge can only connect two nodes. Obviously, this is not suitable for multivariate relations in multi-source heterogeneous data. Therefore, how to use multi-source heterogeneous data to capture multivariate correlations to achieve anomaly detection is crucial to judging the operating status of the system.

[0005] The invention with patent application number CN202410442022.7 proposes a multi-source heterogeneous data prediction method, which is mainly trained according to the following steps: 1) obtain the data stream of each data source; 2) preprocess the obtained data stream to obtain the preprocessed data stream; 3) construct the preprocessed data stream into a heterogeneous graph; 4) construct a multi-source heterogeneous data prediction model and train it; 5) input the heterogeneous graph into the pre-trained multi-source heterogeneous data prediction model for prediction to obtain the prediction result. However, this scheme has the following defects: ① Multi-source heterogeneous data is constructed into a heterogeneous graph. Although the edges in the heterogeneous graph can represent different types of relationships, they are still binary, that is, they can only connect two nodes. To represent multi-node relationships, additional nodes and indirect representations of edges are required, which may complicate the graph structure. ② The Transformer module is used to aggregate nodes and learn node representations, but this method usually relies on a hierarchical attention mechanism to propagate information layer by layer, which may limit the propagation of information of long-distance nodes and ignore the complex interactions between multiple nodes.

[0006] The invention with patent application number CN202110999765.0 proposes a student performance prediction method based on a hypergraph neural network, which is mainly trained according to the following steps: 1) Extract multi-source behavior features based on multi-source heterogeneous data of students; 2) Perform sensitivity analysis on the multi-source behavior features of all students to obtain the influence features of each behavior; 3) Use the influence features to construct a multi-source behavior hypergraph; 4) Predict student performance. However, this scheme has the following defects: ① This method uses a decision tree classification model to obtain the highest influence feature of a single behavior, but this method is usually used for the classification of static data, and it is difficult to process time series data or data with sequential relationships, and it is easy to overfit, especially when the feature dimension is high and the sample size is limited; ② Use multiple hypergraph splicing methods to form the final hypergraph. This method may lose important information during the splicing process and cannot flexibly fuse multiple hypergraphs.

[0007] The shortcomings of the above-mentioned prior art can be summarized as follows: in multi-source heterogeneous data, the data is processed by constructing a heterogeneous graph structure, but the relationship represented by the traditional graph structure edge is still binary, which cannot well represent the multivariate relationship of multi-source heterogeneous data, and cannot well process some time series data with high-dimensional features or data with sequential relationships, which is prone to overfitting. Summary of the invention

[0008] In view of the deficiencies in the prior art, the object of the present invention is to provide a method and system for anomaly detection in an electric power Internet of Things based on hypergraph fusion.

[0009] In order to achieve the above object, the present invention provides the following technical solution: a method for detecting anomalies in the power Internet of Things based on hypergraph fusion, the detection method comprising the following steps: Capture multi-source heterogeneous data in the power Internet of Things and pre-process the acquired multi-source heterogeneous data; The preprocessed multi-source heterogeneous data are input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, and adversarial training is added in the training process of encoder-decoder; Perform cluster analysis on the learned latent feature representations and construct a hypergraph based on the analysis results; The local structural information of each hypergraph is extracted through HGCN, and multiple hypergraphs are weightedly fused through the Transformer encoder. The hypergraph feature representation is learned by the hypergraph attention mechanism for the fused hypergraph. The output of the hypergraph attention mechanism is predicted using the Transformer encoder and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal, thus completing anomaly detection.

[0010] In a preferred embodiment, capturing multi-source heterogeneous data in the power Internet of Things and preprocessing the acquired multi-source heterogeneous data includes the following steps: Collect data streams from different data sources, including power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, and divide them into test sets and training sets. Perform data cleaning, data standardization, and data format conversion preprocessing on the collected data. Data standardization and data format conversion are applicable to both training sets and test sets, while data cleaning is only applicable to training sets. Data cleaning: Different methods are used to process missing values ​​and duplicate values ​​for different types of data streams. Among them, power equipment sensor data and environmental data are both time series data. Linear interpolation is used to fill missing values. For duplicate values, timestamp duplication is checked and duplicate timestamp records are deleted. For missing values ​​of network traffic data, the mean is used to fill, and duplicate values ​​are deleted by deleting duplicate data packets. For missing values ​​of log data, they are skipped directly, and duplicate values ​​are deleted by deleting duplicate log records. For missing values ​​of user behavior data, the mode is used to fill, and duplicate values ​​are deleted by deleting duplicate user operation records. Data standardization: Data after data cleaning is normalized using Min-Max normalization. The formula is as follows: , in They represent the original eigenvalue, the normalized eigenvalue, the minimum value in the original feature, and the maximum value in the original feature respectively.

[0011] In a preferred embodiment, the preprocessed multi-source heterogeneous data are respectively input into an encoder-decoder composed of an LSTM-Transformer encoder and a Transformer encoder-LSTM according to different data types to learn the potential feature representation of the multi-source heterogeneous data, including the following steps: According to the different data types of power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, they are input into two encoder-decoders; among them, the power equipment sensor data, network traffic data, and environmental data are input into the encoder-decoder composed of the LSTM-Transformer encoder, and the log data and user behavior data are input into the encoder-decoder composed of the Transformer encoder-LSTM, and the potential feature representations are learned respectively.

[0012] In a preferred embodiment, adversarial training is added to the training process of the encoder-decoder, comprising the following steps: In the process of encoder-decoder training, a discriminator is added for adversarial training. The discriminator uses a convolutional neural network CNN. In the adversarial training process, the generator is an encoder-decoder composed of LSTM and Transformer encoders. In the training process, there are the loss function of the discriminator and the adversarial loss function of the generator. The loss function of the discriminator is as follows: , in For real data, is the probability distribution of the real data, For expectations, The fake data generated by the generator is composed of random noise. generate, is the probability distribution of random noise, For the discriminator to the real data The output of is the probability of true data, Fake data generated by the generator for the discriminator The output of is the probability of judging whether the generated data is from the real data. Represents the loss of the discriminator on real data, which the discriminator hopes to maximize , Represents the loss of the discriminator on the generated data, which the discriminator hopes to maximize ; The goal of the generator is to generate data that can deceive the discriminator. The adversarial loss function of the generator is as follows: , The generator hopes to maximize , that is, the generated fake data The discriminator determines it as real data; The overall loss function is as follows: , in Represents the weight parameter.

[0013] In a preferred embodiment, cluster analysis is performed on the learned latent feature representations, and a hypergraph is constructed based on the analysis results, including the following steps: The hypergraph is constructed by clustering the latent feature representation of the encoder-decoder. Mini-Batch K-Means is used to cluster the latent feature representation. The formula is described as follows: , in represents a set of clusters, K represents the number of clusters, represents the sample set of the kth cluster in the tth iteration, The potential feature representation belonging to the current batch of data, represents the centroid of the kth cluster in the tth iteration; after clustering, it is expressed as: , H k is the set of potential feature representations belonging to the k-th cluster, represents the kth cluster; The hypergraph is defined as ,in represents the vertex, corresponding to the latent feature representation of the data, Represents a hyperedge, each hyperedge connects multiple vertices, indicating the multi-source relationship between them; construct a hypergraph based on the clustering results, and the centroid of each cluster As the vertices of the hypergraph, ,If the Euclidean clustering between the centroids of two clusters is less than the set threshold, a hyperedge is constructed between them.

[0014] In a preferred embodiment, the local structural information of each hypergraph is extracted by HGCN, and multiple hypergraphs are weightedly fused by Transformer encoder, including the following steps: In HGCN, the hypergraph convolution operation is based on the hypergraph Laplacian matrix, and the hypergraph Laplacian matrix expression is: , Where I represents the identity matrix, represents the vertex degree matrix, represents the hyperedge degree matrix, that is, the degree of each hyperedge, represents the weight matrix of the hyperedge of the a-th hypergraph, represents the transpose of the weight matrix of the hyperedge of the a-th hypergraph, represents the a-th hypergraph adjacency matrix, represents the transpose of the a-th hypergraph adjacency matrix, for -dimensional matrix, where p is the number of vertices, is the number of hyperedges in the a-th hypergraph; HGCN performs hypergraph convolution on a single hypergraph to aggregate node features and obtain new features after convolution. The new features are input into the Transformer encoder to learn the weight of each hypergraph and perform weighted fusion on multiple hypergraphs through the generated weights. The formula is as follows: , in Represents the value matrix corresponding to the fused hypergraph features, represents the number of hypergraphs, represents the weight of the a-th hypergraph, Represents the value matrix corresponding to the a-th hypergraph feature.

[0015] In a preferred embodiment, a hypergraph feature representation is learned by using a hypergraph attention mechanism on the fused hypergraph, comprising the following steps: Node-level attention and edge-level attention are introduced to learn hypergraph feature representation for the fused hypergraph; The node-level attention learning is based on the importance of the hyperedge neighbors to each node in the heterogeneous graph, and the representation of the neighbors is aggregated to form the node embedding; the node-level attention formula is as follows: , in Indicates that on the edge The attention weight between node r and node q is Represent the feature vectors of node r and node q respectively, Represents a hyperedge The feature vector of , W represents the linear transformation matrix, which is used to linearly transform the features of nodes and hyperedges. represents the transpose of the vector used to calculate the attention weights, represents the LeakyReLU activation function, || represents the concatenation operation of the vector, Representation and Hyperedge The collection of all related nodes, Representation and Hyperedge Nodes in all related node sets The feature vector of , and then update the node features according to the correlation coefficient. The formula is as follows: , in Indicates that on the edge The feature representation of node r after aggregation based on its neighbor nodes; finally, the node embedding for different types of hyperedges is , Indicates the hyperedge type; The calculation formula of edge-level attention is as follows: , in represents the normalization of the softmax function, Indicates the hyperedge type The edge-level attention weights, Indicates that node r is in the hyperedge type The following feature representation: represents the transpose of the vector used to compute edge-level attention, represents the LeakyReLU activation function, Represents the total number of nodes in the graph and is used to average the features of the nodes. represents the set of all hyperedge types; Final hypergraph feature representation , the formula is as follows: , in Indicates the total number of hyperedge types.

[0016] In a preferred embodiment, the output of the hypergraph attention mechanism is predicted using a Transformer encoder, and reconstructed using a variational autoencoder VAE, and an abnormality score is calculated using the predicted value and the reconstruction probability, a threshold is set, and the abnormality score is compared with the threshold to determine whether it is abnormal, including the following steps: The Transformer encoder is used as the prediction model, the final hypergraph feature representation is used as the input of the Transformer encoder, and the root mean square error is used as the loss function: , in represent the true value and the predicted value respectively. Indicates a timestamp; Variational Autoencoder (VAE) is used as the reconstruction model to capture the data distribution of the entire data stream by treating the values ​​represented by the hypergraph features as variables. The final loss is: ,in Indicates the loss of reconstruction; The anomaly score of the timestamp is calculated by the following formula: , in is the reconstruction probability, is the predicted value, is a learnable hyperparameter; the threshold is automatically selected using the peak thresholding method, and timestamps with anomaly scores greater than the threshold are considered anomalies.

[0017] A power Internet of Things anomaly detection system based on hypergraph fusion, comprising a preprocessing module, a potential feature extraction module, a hypergraph construction module, a weighted fusion module, a hypergraph feature representation learning module, and an anomaly detection module; Preprocessing module: captures multi-source heterogeneous data in the power Internet of Things and preprocesses the acquired multi-source heterogeneous data; Potential feature extraction module: The preprocessed multi-source heterogeneous data is input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, and adversarial training is added in the training process of encoder-decoder; Hypergraph construction module: cluster analysis is performed on the learned latent feature representations, and a hypergraph is constructed based on the analysis results; Weighted fusion module: extract the local structural information of each hypergraph through HGCN, and perform weighted fusion of multiple hypergraphs through Transformer encoder; Hypergraph feature representation learning module: uses the hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph; Anomaly detection module: The output of the hypergraph attention mechanism is predicted using the Transformer encoder, and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal and complete anomaly detection.

[0018] In the above technical solution, the technical effects and advantages provided by the present invention are: 1. The present invention introduces a hypergraph neural network into the task of abnormal detection of the power Internet of Things. The hypergraph can model multivariate relationships more accurately. By constructing a hypergraph, the multivariate relationships of multi-source heterogeneous data are represented. Unlike the ordinary graph structure, the hyperedge of the hypergraph can connect multiple vertices. Such a hypergraph structure can better model multi-source heterogeneous data to dig out the relationship between multi-source heterogeneous data, so that multi-source heterogeneous data can be processed more flexibly. Therefore, the present invention adopts the construction of an LSTM-Transformer model as an encoder-decoder for feature extraction of multi-source heterogeneous data. The representation of low-dimensional features can be better learned by reconstruction, and the strategy of adversarial training is introduced in the reconstruction stage, so that low-dimensional features can be better learned. The clustering results represented by low-dimensional features are used to construct a hypergraph of multi-source heterogeneous data to dig out the complex and diverse relationships between multi-source heterogeneous data.

[0019] 2. The present invention adopts the HGCN-Transformer method to perform weighted fusion on the learned multiple hypergraph structures. HGCN-Transformer can dynamically adjust the weights of each hypergraph according to the network status and data, so as to better adapt to environmental changes and provide real-time analysis results. It also uses the hypergraph attention mechanism to learn the representation of hypergraph structure nodes, helping to improve the accuracy of anomaly detection.

[0020] 3. The present invention adopts joint training, which takes into account both the periodic information of the time series and the global data distribution of the time series when judging abnormal sequences. Joint training can combine the advantages of the prediction model and the reconstruction model, while perfectly complementing their shortcomings. In addition, Transformer is introduced into the prediction model, and Transformer can effectively capture the information of the global context, helping to improve the accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0022] Figure 1 is a flow chart of the method of the present invention; Figure 2 The figure is a diagram showing the overall structure of the method of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Embodiment 1: The scale of the power Internet of Things and the complex technical infrastructure with higher levels of automation, connectivity and remote access require the use of optimal and intelligent methods to detect various anomalies. In addition, the multi-source heterogeneous data in the power Internet of Things is complex and diverse, and a large amount of complex data will be generated from different sources and different types of equipment. Therefore, in the power Internet of Things, how to effectively process multi-source heterogeneous data becomes a major challenge. In this regard, this embodiment provides a power Internet of Things anomaly detection method based on hypergraph fusion to improve the accuracy and speed of detection and reduce the risk of missed reports and false alarms. Unlike ordinary graph structures, the hyperedges of hypergraphs can connect multiple vertices. Such hypergraph structures can better model multi-source heterogeneous data to mine the relationship between multi-source heterogeneous data, so that multi-source heterogeneous data can be processed more flexibly. Therefore, this embodiment adopts the construction of LSTM-Transformer model as the encoder-decoder for multi-source heterogeneous data feature extraction, and can better learn the representation of low-dimensional features through reconstruction, and introduce adversarial training strategies in the reconstruction stage, so that low-dimensional features can be better learned. The clustering results represented by low-dimensional features are used to construct a hypergraph of multi-source heterogeneous data to explore the complex and diverse relationships between multi-source heterogeneous data.

[0025] If the learned multiple hypergraph structures cannot be well fused, some important relationship features may be lost. In addition, the data flow in the power Internet of Things is usually dynamically changing, which causes the topological structure of the hypergraph to also change dynamically. It is necessary to effectively fuse the dynamically changing hypergraph structure. In addition, it is necessary to aggregate and update the node information of the learned hypergraph structure to better learn the representation of the nodes, so as to improve the accuracy of anomaly detection. Therefore, this embodiment adopts the HGCN-Transformer method to perform weighted fusion of the learned multiple hypergraph structures. HGCN-Transformer can dynamically adjust the weights of each hypergraph according to the network status and data, so as to better adapt to environmental changes, provide real-time analysis results, and use the hypergraph attention mechanism to learn the representation of the hypergraph structure nodes, helping to improve the accuracy of anomaly detection.

[0026] See also Figure 1 and Figure 2As shown, the present embodiment describes a method for detecting anomalies in the power Internet of Things based on hypergraph fusion, and the detection method comprises the following steps: (1) Capture multi-source heterogeneous data in the power Internet of Things and pre-process the acquired multi-source heterogeneous data. Data cleaning and data standardization are used to pre-process the data, scale the attributes of multi-source heterogeneous data to a certain range, and replace abnormal values ​​in the data with normal values.

[0027] (2) The preprocessed multi-source heterogeneous data are input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data. The combination of LSTM and Transformer allows the model to flexibly process different types of data. LSTM can focus on data with strong time series (power equipment sensor data, network traffic data, environmental data), while Transformer can process more types of heterogeneous data (log data, user behavior data). Therefore, the encoder-decoder can map data from different sources to the same feature space, which is convenient for comprehensive analysis and modeling. Therefore, the data with strong time series is input into the encoder-decoder composed of LSTM-Transformer encoder, and the other two heterogeneous data are input into the encoder-decoder composed of Transformer encoder-LSTM. In addition, adding adversarial training to the training process of the encoder-decoder can enhance the robustness and generalization ability of the model. Adversarial learning can help the model extract robust features more effectively when facing noise and data inconsistency.

[0028] (3) Perform cluster analysis on the learned latent feature representations and construct a hypergraph based on the analysis results. The feature representations are used as nodes, and hyperedges are constructed based on the clustering results. The nodes in each cluster form a hyperedge, thereby capturing the complex relationship between multiple samples in the feature representation. The present invention uses Mini-Batch K-Means to perform latent feature representation cluster analysis. This method updates the cluster center by using a small portion of data instead of the entire data set. This significantly reduces memory requirements, allowing it to run in a memory-constrained environment, and speeds up the operation to provide reasonable clustering results in a shorter time.

[0029] (4) Extract the local structural information of each hypergraph through HGCN, perform weighted fusion of multiple hypergraphs through Transformer encoder, and use hypergraph attention mechanism to learn hypergraph feature representation for the fused hypergraph; (5) A prediction and reconstruction method is used for joint training. The output of the hypergraph attention mechanism is predicted using a Transformer encoder and reconstructed using a variational autoencoder (VAE). The predicted value and the reconstruction probability are used to calculate the anomaly score. A threshold is set and the anomaly score is compared with the threshold to determine whether the multivariate time series is abnormal, thus completing anomaly detection.

[0030] Capture multi-source heterogeneous data in the power Internet of Things and pre-process the acquired multi-source heterogeneous data, including: Collect data streams from different data sources, including power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, and divide them into test sets and training sets. Due to the multi-source and heterogeneous nature of data streams, in order to improve the stability of the model, the collected data is preprocessed by data cleaning, data standardization, and data format conversion. Data standardization and data format conversion are applicable to both training sets and test sets, while data cleaning is only applicable to training sets.

[0031] Data cleaning: Different methods are used to handle missing values ​​and duplicate values ​​for different types of data streams. Among them, power equipment sensor data and environmental data are time series data, and there may be problems with missing or repeated sensor readings, so linear interpolation is used to fill missing values. For duplicate values, timestamp duplication is checked and duplicate timestamp records are deleted. Missing values ​​in network traffic data are filled with the mean, and duplicate values ​​are deleted by deleting duplicate data packets, especially duplicate rows based on features such as timestamps, source IPs, and destination IPs. Missing values ​​in log data can be skipped directly, and duplicate values ​​are deleted by deleting duplicate log records. Missing values ​​in user behavior data are filled with the mode, and duplicate values ​​are deleted by deleting duplicate user operation records.

[0032] Data standardization: Data is normalized after data cleaning, and the data is scaled to a certain range. The purpose of standardization is to eliminate the differences in data features so that the effects of different features on the model are roughly the same. Min-Max normalization is used in this invention, and the formula is as follows: , in They represent the original eigenvalue, the normalized eigenvalue, the minimum value in the original feature, and the maximum value in the original feature respectively.

[0033] The preprocessed multi-source heterogeneous data are input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, including: The preprocessed data stream is input into the encoder-decoder composed of LSTM and Transformer to extract potential features. By adding adversarial learning to the encoder-decoder architecture, the model will be trained to not only focus on easy-to-learn features, but also be able to cope with potential noise or adversarial perturbations. For multi-source heterogeneous data, this method can enable the model to learn more robust potential feature representations, not just limited to a specific data source or a specific type of feature.

[0034] LSTM is composed of multiple layers stacked together. The core parts of LSTM are forget gate, input gate and output gate. The forget gate inputs the information of the hidden state of the previous moment and the data of the current moment into the Sigmoid function. The output value is between 0 and 1, indicating whether it should be forgotten or not. The closer to 0, the more it should be forgotten, and the closer to 1, the more it should be remembered. The final value will be multiplied by the memory of the previous moment to limit the influence of the previous memory on the later. The input gate controls how much input information of the current time step will be updated to the cell state. The output gate controls the output of the hidden state of the current time step. The hidden state is the final output of LSTM. The output gate determines which parts of the cell state will affect the hidden state, thereby determining the output content of the current time step. LSTM is suitable for processing time series data, especially data with time dependence. It can effectively capture short-term and long-term dependency patterns in the data, so it is suitable for processing data sources with strong time series. Finally, LSTM will output the hidden state of each source data.

[0035] Transformer consists of an encoder and a decoder. The encoder is responsible for processing the input data and converting it into an internal representation that the model can understand, while the decoder is responsible for generating output data based on this internal representation. Tranformer can be used to process more types of heterogeneous data. In this invention, only the Tranformer encoder is used.

[0036] According to the different data types of power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, they are input into two encoder-decoders; among them, the power equipment sensor data, network traffic data, and environmental data are input into the encoder-decoder composed of the LSTM-Transformer encoder, and the log data and user behavior data are input into the encoder-decoder composed of the Transformer encoder-LSTM, and the potential feature representations are learned respectively.

[0037] Add adversarial training to the encoder-decoder training process, including: In the training process of the encoder-decoder, the discriminator is added for adversarial training, so that better potential feature representation can be obtained. The discriminator uses a convolutional neural network (CNN), which includes the output of the convolution layer, ReLU activation layer, maximum pooling layer, batch normalization layer, and fully connected layer, and the output layer is activated by Sigmoid. The generator in the adversarial training process is an encoder-decoder composed of LSTM and Transformer encoders. Therefore, there are loss functions of the discriminator and adversarial loss functions of the generator during the training process. The loss function of the discriminator is as follows: , in For real data, is the probability distribution of the real data, For expectations, The fake data generated by the generator is composed of random noise. generate, is the probability distribution of random noise, For the discriminator to the real data The output of is the probability of true data, Fake data generated by the generator for the discriminator The output of is the probability of judging whether the generated data is from the real data. Represents the loss of the discriminator on real data, which the discriminator hopes to maximize , that is, we hope that the real data will be judged as real data (with a probability close to 1). Represents the loss of the discriminator on the generated data, which the discriminator hopes to maximize , that is, we hope that the generated data will be judged as false data (with a probability close to 0). By minimizing the loss of the discriminator, the discriminator continuously improves its ability to distinguish between real data and generated data.

[0038] The goal of the generator is to generate data that can deceive the discriminator, so its loss function is defined as hoping that the generated data will be judged as real data by the discriminator, that is, hoping that the output of the discriminator is close to 1. The adversarial loss function of the generator is as follows: , The generator hopes to maximize , that is, the generated fake data The discriminator judges it as real data (with a probability close to 1). By minimizing the adversarial loss of the generator, the generator continuously adjusts its parameters to make the generated data more realistic, thereby deceiving the discriminator.

[0039] The overall loss function is as follows: , in Represents the weight parameter.

[0040] Perform cluster analysis on the learned latent feature representations and construct a hypergraph based on the analysis results, including: The hypergraph is constructed by clustering the potential feature representation of the encoder-decoder. In this embodiment, Mini-Batch K-Means is used to cluster the potential feature representation. The formula is described as follows: , in represents a set of clusters, K represents the number of clusters, represents the sample set of the kth cluster in the tth iteration (this only contains the samples selected from the Mini-Batch in the tth iteration), The potential feature representation belonging to the current batch of data, represents the centroid of the kth cluster in the tth iteration; after clustering, it can be expressed as: , H k is the set of potential feature representations belonging to the k-th cluster, represents the kth cluster; the features in each cluster share certain similarities, which can be seen as the local structure of the data.

[0041] Hypergraph is a generalized form of graph that can represent multi-source relationships. Edges in ordinary graphs can only connect two vertices, while hyperedges in hypergraphs can connect any number of vertices. Therefore, hypergraphs have stronger expressive power when dealing with complex relationships. Hypergraphs can be defined as ,in represents the vertex, corresponding to the latent feature representation of the data, Represents a hyperedge, each hyperedge connects multiple vertices, indicating the multi-source relationship between them. Construct a hypergraph based on the clustering results. The specific steps are as follows: The centroid of each cluster can be used as the vertices of the hypergraph, that is ,If the Euclidean clustering between the centroids of two clusters is less than the set threshold, a hyperedge is constructed between them.

[0042] The local structural information of each hypergraph is extracted through HGCN, and multiple hypergraphs are weightedly fused through the Transformer encoder, including: After the hypergraph is constructed, multiple hypergraphs need to be weighted and fused. The local structural information of each hypergraph is extracted through HGCN, and multiple hypergraphs are weighted and fused through the Transformer encoder, which can make full use of the relationship between multiple hypergraphs to improve the representation ability. And the weights can be adaptively allocated according to different graph structures, making the fusion of multiple hypergraphs more flexible.

[0043] In HGCN, the hypergraph convolution operation is based on the hypergraph Laplacian matrix, and the hypergraph Laplacian matrix expression is: , Where I represents the identity matrix, represents the vertex degree matrix, represents the hyperedge degree matrix, that is, the degree of each hyperedge, represents the weight matrix of the hyperedge of the a-th hypergraph, represents the transpose of the weight matrix of the hyperedge of the a-th hypergraph, represents the a-th hypergraph adjacency matrix, represents the transpose of the a-th hypergraph adjacency matrix, for -dimensional matrix, where p is the number of vertices, is the number of hyperedges in the a-th hypergraph.

[0044] HGCN performs hypergraph convolution on a single hypergraph to aggregate node features and obtain new features after convolution. The new features are input into the Transformer encoder to learn the weight of each hypergraph and perform weighted fusion on multiple hypergraphs through the generated weights. The formula is as follows: , in Represents the value matrix corresponding to the fused hypergraph features, represents the number of hypergraphs, represents the weight of the a-th hypergraph, Represents the value matrix corresponding to the a-th hypergraph feature.

[0045] The hypergraph attention mechanism is used to learn the hypergraph feature representation of the fused hypergraph, including: Node-level attention and edge-level attention are introduced to learn hypergraph feature representation for the fused hypergraph; The node-level attention can learn the importance of hyperedge-based neighbors to each node in the heterogeneous graph and aggregate the representations of these meaningful neighbors to form node embeddings; the node-level attention formula is as follows: , in Indicates that on the edge The attention weight between node r and node q is Represent the feature vectors of node r and node q respectively, Represents a hyperedge The feature vector of (aggregated based on the connected node features), W represents the linear transformation matrix, which is used to linearly transform the features of nodes and hyperedges. represents the transpose of the vector used to calculate the attention weights, represents the LeakyReLU activation function, || represents the concatenation operation of the vector, Representation and Hyperedge The collection of all related nodes, Representation and Hyperedge Nodes in all related node sets The feature vector of , and then the node features can be updated according to the correlation coefficient. The formula is as follows: , in Indicates that on the edge The feature representation of node r after aggregation based on its neighbor nodes; finally, the node embedding for different types of hyperedges is , Indicates the hyperedge type.

[0046] Edge-level attention needs to consider the weight differences between different hyperedge types. The calculation formula for edge-level attention is as follows: , in Indicates the normalization of the softmax function. The function of the softmax function is to convert a set of input values ​​into a probability distribution through exponential operation and normalization; Indicates the hyperedge type The edge-level attention weights, Indicates that node r is in the hyperedge type The following feature representation: represents the transpose of the vector used to compute edge-level attention, LeakyReLU activation function, which introduces a small non-zero slope in the negative region to avoid the complete failure of the ReLU activation function when the input is negative, thereby alleviating the phenomenon of "neuron death"; Represents the total number of nodes in the graph and is used to average the features of the nodes. Represents the set of all hyperedge types.

[0047] Get the final hypergraph feature representation , the formula is as follows: , in Indicates the total number of hyperedge types.

[0048] The output of the hypergraph attention mechanism is predicted using the Transformer encoder, and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal. The following steps are included: The Transformer encoder is used as the prediction model, the final hypergraph feature representation is used as the input of the Transformer encoder, and the root mean square error is used as the loss function: , in represent the true value and the predicted value respectively. Indicates a timestamp; Variational autoencoder (VAE) is used as the reconstruction model, which provides a probabilistic way to describe observations in the latent space. By treating the values ​​represented by the hypergraph features as variables, the data distribution of the entire data stream can be captured.

[0049] So the final loss is: ,in Represents the reconstruction loss.

[0050] The anomaly score of the timestamp is calculated by the following formula: , in is the reconstruction probability, is the predicted value, is a learnable hyperparameter; this embodiment uses the peak over-threshold method to automatically select the threshold, and regards the timestamp with anomaly score greater than the threshold as anomaly.

[0051] Embodiment 2: This embodiment provides a power Internet of Things anomaly detection system based on hypergraph fusion, including a preprocessing module, a potential feature extraction module, a hypergraph construction module, a weighted fusion module, a hypergraph feature representation learning module, and an anomaly detection module; Preprocessing module: captures multi-source heterogeneous data in the power Internet of Things and preprocesses the acquired multi-source heterogeneous data; Potential feature extraction module: The preprocessed multi-source heterogeneous data is input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, and adversarial training is added in the training process of encoder-decoder; Hypergraph construction module: cluster analysis is performed on the learned latent feature representations, and a hypergraph is constructed based on the analysis results; Weighted fusion module: extract the local structural information of each hypergraph through HGCN, and perform weighted fusion of multiple hypergraphs through Transformer encoder; Hypergraph feature representation learning module: uses the hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph; Anomaly detection module: The output of the hypergraph attention mechanism is predicted using the Transformer encoder, and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal and complete anomaly detection.

[0052] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0053] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for detecting anomalies in the power Internet of Things based on hypergraph fusion, characterized in that: The detection method comprises the following steps: Capture multi-source heterogeneous data in the power Internet of Things and pre-process the acquired multi-source heterogeneous data; The preprocessed multi-source heterogeneous data are input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, and adversarial training is added in the training process of encoder-decoder; Perform cluster analysis on the learned latent feature representations and construct a hypergraph based on the analysis results; The local structural information of each hypergraph is extracted through HGCN, and multiple hypergraphs are weightedly fused through the Transformer encoder. The hypergraph feature representation is learned by the hypergraph attention mechanism for the fused hypergraph. The output of the hypergraph attention mechanism is predicted using the Transformer encoder and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal, thus completing anomaly detection.

2. According to claim 1, a method for detecting anomalies in the power Internet of Things based on hypergraph fusion is characterized in that: Capturing multi-source heterogeneous data in the power Internet of Things and preprocessing the acquired multi-source heterogeneous data includes the following steps: Collect data streams from different data sources, including power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, and divide them into test sets and training sets. Perform data cleaning, data standardization, and data format conversion preprocessing on the collected data. Data standardization and data format conversion are applicable to both training sets and test sets, while data cleaning is only applicable to training sets. Data cleaning: Different methods are used to process missing values ​​and duplicate values ​​for different types of data streams. Among them, power equipment sensor data and environmental data are both time series data. Linear interpolation is used to fill missing values. For duplicate values, timestamp duplication is checked and duplicate timestamp records are deleted. For missing values ​​of network traffic data, the mean is used to fill, and duplicate values ​​are deleted by deleting duplicate data packets. For missing values ​​of log data, they are skipped directly, and duplicate values ​​are deleted by deleting duplicate log records. For missing values ​​of user behavior data, the mode is used to fill, and duplicate values ​​are deleted by deleting duplicate user operation records. Data standardization: Data after data cleaning is normalized using Min-Max normalization. The formula is as follows: ， in They represent the original eigenvalue, the normalized eigenvalue, the minimum value in the original feature, and the maximum value in the original feature respectively.

3. According to claim 2, a method for detecting anomalies in the power Internet of Things based on hypergraph fusion is characterized in that: The preprocessed multi-source heterogeneous data are input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, including the following steps: According to the different data types of power equipment sensor data, network traffic data, environmental data, log data, and user behavior data, they are input into two encoder-decoders; among them, the power equipment sensor data, network traffic data, and environmental data are input into the encoder-decoder composed of the LSTM-Transformer encoder, and the log data and user behavior data are input into the encoder-decoder composed of the Transformer encoder-LSTM, and the potential feature representations are learned respectively.

4. The method for detecting anomalies in the power Internet of Things based on hypergraph fusion according to claim 3 is characterized in that: Adding adversarial training to the encoder-decoder training process includes the following steps: In the process of encoder-decoder training, a discriminator is added for adversarial training. The discriminator uses a convolutional neural network CNN. In the adversarial training process, the generator is an encoder-decoder composed of LSTM and Transformer encoders. In the training process, there are the loss function of the discriminator and the adversarial loss function of the generator. The loss function of the discriminator is as follows: ， in For real data, is the probability distribution of the real data, For expectations, The fake data generated by the generator is composed of random noise. generate, is the probability distribution of random noise, For the discriminator to the real data The output of is the probability of true data, Fake data generated by the generator for the discriminator The output of is the probability of judging whether the generated data is from the real data. Represents the loss of the discriminator on real data, which the discriminator hopes to maximize , Represents the loss of the discriminator on the generated data, which the discriminator hopes to maximize ; The goal of the generator is to generate data that can deceive the discriminator. The adversarial loss function of the generator is as follows: ， The generator hopes to maximize , that is, the generated fake data The discriminator determines it as real data; The overall loss function is as follows: ， in Represents the weight parameter.

5. According to claim 4, a method for detecting anomalies in the power Internet of Things based on hypergraph fusion is characterized in that: Perform cluster analysis on the learned latent feature representations and construct a hypergraph based on the analysis results, including the following steps: The latent feature representation of the encoder-decoder is clustered to construct a hypergraph, and Mini-Batch K-Means is used to cluster the latent feature representation. The formula is described as follows: ， in represents a set of clusters, K represents the number of clusters, represents the sample set of the kth cluster in the tth iteration, The potential feature representation belonging to the current batch of data, represents the centroid of the kth cluster in the tth iteration; after clustering, it is expressed as: ， H k is the set of potential feature representations belonging to the k-th cluster, represents the kth cluster; The hypergraph is defined as ,in represents the vertex, corresponding to the latent feature representation of the data, Represents a hyperedge, each hyperedge connects multiple vertices, indicating the multi-source relationship between them; construct a hypergraph based on the clustering results, and the centroid of each cluster As the vertices of the hypergraph, ,If the Euclidean clustering between the centroids of two clusters is less than the set threshold, a hyperedge is constructed between them.

6. The method for detecting anomalies in the power Internet of Things based on hypergraph fusion according to claim 5 is characterized in that: The local structural information of each hypergraph is extracted through HGCN, and multiple hypergraphs are weightedly fused through the Transformer encoder, including the following steps: In HGCN, the hypergraph convolution operation is based on the hypergraph Laplacian matrix, and the hypergraph Laplacian matrix expression is: ， Where I represents the identity matrix, represents the vertex degree matrix, represents the hyperedge degree matrix, that is, the degree of each hyperedge, represents the weight matrix of the hyperedge of the a-th hypergraph, represents the transpose of the weight matrix of the hyperedge of the a-th hypergraph, represents the a-th hypergraph adjacency matrix, represents the transpose of the a-th hypergraph adjacency matrix, for -dimensional matrix, where p is the number of vertices, is the number of hyperedges in the a-th hypergraph; HGCN performs hypergraph convolution on a single hypergraph to aggregate node features and obtain new features after convolution. The new features are input into the Transformer encoder to learn the weight of each hypergraph and perform weighted fusion on multiple hypergraphs through the generated weights. The formula is as follows: ， in Represents the value matrix corresponding to the fused hypergraph features, represents the number of hypergraphs, represents the weight of the a-th hypergraph, Represents the value matrix corresponding to the a-th hypergraph feature.

7. The method for detecting anomalies in the power Internet of Things based on hypergraph fusion according to claim 6 is characterized by: The hypergraph feature representation is learned by using the hypergraph attention mechanism on the fused hypergraph, including the following steps: Node-level attention and edge-level attention are introduced to learn hypergraph feature representation for the fused hypergraph; The node-level attention learning is based on the importance of the hyperedge neighbors to each node in the heterogeneous graph, and the representation of the neighbors is aggregated to form the node embedding; the node-level attention formula is as follows: ， in Indicates that on the edge The attention weight between node r and node q is Represent the feature vectors of node r and node q respectively, Represents a hyperedge The feature vector of , W represents the linear transformation matrix, which is used to linearly transform the features of nodes and hyperedges. represents the transpose of the vector used to calculate the attention weights, represents the LeakyReLU activation function, || represents the concatenation operation of the vector, Representation and Hyperedge The collection of all related nodes, Representation and Hyperedge Nodes in the collection of all related nodes The feature vector of , and then update the node features according to the correlation coefficient. The formula is as follows: ， in Indicates that on the edge The feature representation of node r after aggregation based on its neighbor nodes; finally, the node embedding for different types of hyperedges is , Indicates the hyperedge type; The calculation formula of edge-level attention is as follows: ， in represents the normalization of the softmax function, Indicates the hyperedge type The edge-level attention weights, Indicates that node r is in the hyperedge type The following feature representation: represents the transpose of the vector used to compute edge-level attention, represents the LeakyReLU activation function, Represents the total number of nodes in the graph and is used to average the features of the nodes. represents the set of all hyperedge types; Final hypergraph feature representation , the formula is as follows: ， in Indicates the total number of hyperedge types.

8. The method for detecting anomalies in the power Internet of Things based on hypergraph fusion according to claim 7 is characterized in that: The output of the hypergraph attention mechanism is predicted using the Transformer encoder, and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal. The following steps are included: The Transformer encoder is used as the prediction model, the final hypergraph feature representation is used as the input of the Transformer encoder, and the root mean square error is used as the loss function: ， in represent the true value and the predicted value respectively. Indicates a timestamp; Variational Autoencoder (VAE) is used as the reconstruction model to capture the data distribution of the entire data stream by treating the values ​​represented by the hypergraph features as variables. The final loss is: ,in Indicates the loss of reconstruction; The anomaly score of the timestamp is calculated by the following formula: ， in is the reconstruction probability, is the predicted value, is a learnable hyperparameter; the threshold is automatically selected using the peak thresholding method, and timestamps with anomaly scores greater than the threshold are considered anomalies.

9. A power Internet of Things anomaly detection system based on hypergraph fusion, used to implement the detection method according to any one of claims 1 to 8, characterized in that: It includes preprocessing module, potential feature extraction module, hypergraph construction module, weighted fusion module, hypergraph feature representation learning module and anomaly detection module; Preprocessing module: captures multi-source heterogeneous data in the power Internet of Things and preprocesses the acquired multi-source heterogeneous data; Potential feature extraction module: The preprocessed multi-source heterogeneous data is input into the encoder-decoder composed of LSTM-Transformer encoder and Transformer encoder-LSTM according to different data types to learn the potential feature representation of multi-source heterogeneous data, and adversarial training is added in the training process of encoder-decoder; Hypergraph construction module: cluster analysis is performed on the learned latent feature representations, and a hypergraph is constructed based on the analysis results; Weighted fusion module: extract the local structural information of each hypergraph through HGCN, and perform weighted fusion of multiple hypergraphs through Transformer encoder; Hypergraph feature representation learning module: uses the hypergraph attention mechanism to learn the hypergraph feature representation of the fused hypergraph; Anomaly detection module: The output of the hypergraph attention mechanism is predicted using the Transformer encoder, and reconstructed using the variational autoencoder VAE. The predicted value and the reconstruction probability are used to calculate the anomaly score, a threshold is set, and the anomaly score is compared with the threshold to determine whether it is abnormal and complete anomaly detection.

Images