Multi-source heterogeneous instrument data fusion and anomaly detection method and system
Through the multi-source heterogeneous instrument data fusion and abnormal detection methods based on graph neural network, the data processing complexity and feature engineering dependence problems in traditional methods are solved, efficient abnormal detection and data fusion are achieved, and the system capability is improved Scalability and adaptability.
Patent Information
- Application Number
- CN202411201858.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
The existing technology has low effect in handling massive, real-time, heterogeneous instrument data abnormal detection, complex preprocessing, feature engineering depends on expert knowledge, is difficult to automate, data fusion effect is limited, and the correlation between multi-source data cannot be fully utilized. information.
Using multi-source heterogeneous instrument data fusion and abnormal detection methods based on graph neural network, low-dimensional embeddings are learned from node and time series data by defining heterogeneous instrument data network diagrams, using graph convolutional neural network and Transformer model. Vectors and important features are combined with graph attention network for data fusion and anomaly detection.
It realizes automated processing of multi-source heterogeneous data, improves the accuracy and efficiency of abnormal detection, enhances data fusion and feature extraction capabilities, improves the scalability and adaptability of the system, and realizes the intelligence of complex logical data management and Visualization of data context relationships.
Smart Images

Figure CN119089355A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of graph structure data processing, and in particular relates to a method and system for multi-source heterogeneous instrument data fusion and anomaly detection. Background Art
[0002] With the rapid development of Industry 4.0 and Internet of Things (IoT) technologies, the types and number of instruments used in industrial production processes have increased dramatically. These instruments usually come from different manufacturers and have different communication protocols and data formats, forming multi-source heterogeneous data. How to effectively fuse these multi-source heterogeneous data and extract valuable information has become an important challenge in the current field of industrial data analysis.
[0003] Traditional data anomaly detection methods mainly rely on human resources such as data experts and business experts for investigation. In recent years, with the advancement of data science and the development of artificial intelligence technology, anomaly detection methods based on data mining and intelligent optimization algorithms have continued to emerge. Wang et al. used a variety of clustering algorithms to detect non-technical losses of electric meters by analyzing records such as average loss and loss change efficiency collected by electric meters, and compared the detection effects of various clustering algorithms; Buzau et al. used a hybrid deep neural network combining long-term and short-term memory networks (LSTM) and multi-layer perceptrons (MLP) for anomaly detection based on differences in electricity consumption behavior. The results showed that its accuracy was higher than other classifiers; in view of the correlation of time series data, Chahla et al. also proposed an anomaly detection method based on long-term and short-term memory networks (LSTM) for anomaly search from time series data, and performed anomaly detection through the prediction error and statistical strategy of observed data.
[0004] Through the above analysis, the problems and defects of the existing technology are as follows: With the continuous expansion of the scale of data construction in various professional fields, the traditional methods are relatively ineffective in processing massive, real-time, heterogeneous instrument data anomaly detection. The main problems include: (1) complex preprocessing, different instrument data formats and communication protocols are different, and the data preprocessing process is complex and time-consuming; (2) feature engineering relies on expert knowledge, it is difficult to automate batch processing operations, and it is easy to miss important features, and the scalability is poor; (3) data fusion effect is limited, and traditional methods are difficult to fully utilize the correlation information between multi-source data.
[0005] Although the existing intelligent optimization anomaly detection algorithm has achieved good detection results, it may ignore some important information in specific dimensions and lacks the means to efficiently manage the topological relationship of instruments in a heterogeneous data environment. It is unable to realize the intelligent management of complex logical data and the visualization and clarification of the contextual relationship of instrument data. Summary of the invention
[0006] In order to overcome the problems existing in the related art, the embodiments disclosed in the present invention provide a method and system for multi-source heterogeneous instrument data fusion and anomaly detection, specifically relating to a multi-source heterogeneous instrument data fusion and anomaly detection framework based on a graph neural network model.
[0007] The technical solution is as follows: a method for multi-source heterogeneous instrument data fusion and anomaly detection, the method comprising:
[0008] S1, by defining a heterogeneous instrument data network graph, the data of different instruments are represented as a graph structure;
[0009] S2, instrument node data similarity learning: using graph convolutional neural networks, learn the low-dimensional embedding vector of each node from neighboring nodes, capture the implicit dependencies between nodes, and achieve deep fusion of multi-source heterogeneous data;
[0010] S3, representation learning of instrument time series data: The Transformer model is used to learn the representation of instrument time series data. By learning the context information of historical data, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the global information and historical information of the instrument data are integrated.
[0011] S4, anomaly detection: The probability of anomaly detection is calculated by transforming the output vector of the graph attention network, and the cross entropy loss function is used for end-to-end training of the graph convolutional neural network, Transformer model and graph attention network model.
[0012] In step S1, a heterogeneous instrument data network diagram is defined, including:
[0013] Define a heterogeneous instrument data network as a heterogeneous instrument data network graph G = (V, E, A, X), where V ∈ R n represents the instrument node space of graph G, R is the real number field, n represents the number of instrument nodes; E represents the edge of graph G; A∈R n*n is the adjacency matrix of graph G; for two instrument nodes v i ,v j ∈V, if edge e ij ∈E, then A ij =1, otherwise A ij =0, A ij is the value of the i-th row and j-th column of the adjacency matrix A; X∈R n*m Represents the attribute information of the instrument node, and m is the dimension of the attribute information.
[0014] In step S2, instrument node data similarity learning includes:
[0015] After obtaining the heterogeneous instrument data network graph, the graph convolutional neural network (GCN) is used to learn the low-dimensional embedding vector of each instrument node from neighboring instrument nodes through convolution and aggregation operators.
[0016] In the i-1 layer, the non-negative matrix factorization method NNMF is used to convert the H output of each layer of the graph convolutional neural network GCN network i Decomposed into two low-dimensional matrices; among them, H i It represents the output of the i-th layer network of GCN, and is also the network input of the i+1th layer;
[0017] The low-dimensional vectors of each layer are concatenated and mapped to n dimensions through a single hidden layer perceptron MLP and output.
[0018] Furthermore, in the graph convolutional neural network GCN, the convolution operations of each layer are as follows:
[0019]
[0020] Among them, H i+1 represents the output of the i+1th layer network of GCN, δ represents the activation function, The diagonal bias matrix represented by Represents the value of the i-th row and j-th column of the diagonal matrix; represents the sum of the adjacency matrix of graph G and the diagonal bias matrix, A represents the adjacency matrix of graph G, and I represents the diagonal bias matrix; W i Represents the set of transformation parameters of the i-th layer network.
[0021] Furthermore, the non-negative matrix factorization method NNMF is used in the i-1 layer to transform the H output of each layer of the GCN network i The expression decomposed into two low-dimensional matrices is:
[0022] H i =U i V i ,sT i ≥0,V i ≥0
[0023] in, U i ∈R n*d , L i represents the output dimension of the i-th layer, d is the output dimension of the NNMF method, and U i Indicates H i A lower dimensional representation vector, V i Indicates H i Another part of matrix factorization;
[0024] Mapped to m dimensions through a single hidden layer perceptron MLP, the final output is expressed as:
[0025] H i ′=MLP(‖U0‖‖U1‖…‖U i-1 ‖)
[0026] Among them, ‖·‖ represents feature concatenation, H i ′ represents the final output vector of the GCN network, MLP() represents the multi-layer perceptron model, U0 represents the low-dimensional representation vector of the 0th layer after matrix decomposition, U1 represents the low-dimensional representation vector of the 1st layer after matrix decomposition, and U i-1 Represents the low-dimensional representation vector of the i-1th layer after matrix decomposition.
[0027] In step S3, the representation learning of instrument time series data includes:
[0028] Based on the historical data of a certain time window, the Transformer attention model is used to learn the context information of the instrument data; the calculation process is as follows:
[0029] Q,K,V=embedding(X)
[0030]
[0031] head i =Attention(QW i Q ,KW i K ,VW i V )
[0032] Multihead(Q,K,V)=concat(head1,…,head n )W o
[0033] in, are all Transformer model parameter vectors; after obtaining the multi-head attention vector representation, the Transformer output τ is further obtained: Q represents the query matrix of the Transformer model, K represents the key matrix of the Transformer model, V represents the value matrix of the Transformer model, embedding(X) represents the mapping model of the instrument node feature X, Attention() represents the attention calculation operation of the Transformer model, softmax() represents the softmax mathematical calculation operation, R represents the matrix transpose operation, d kRepresents the dimension of the K matrix in Transformer, head i represents the i-th output head of the Transformer model, W i Represents the W matrix of the i-th layer of the Transformer model, Multihead() represents the calculation operation of multiple output heads of the Transformer model, concat() represents the vector concatenation operation, head1 represents the first output head of the Transformer model, head n represents the nth output head of the Transformer model, W o Represents the matrix representation of the output layer of the Transformer model;
[0034] H1=LN(Mutlihead(Q,K,V)+X)
[0035] H2=FFN(H1)
[0036] τ=LN(H1+H2)
[0037] Among them, LN(·) represents the layer normalization method, FFN(·) represents the feedforward neural network layer, H1 and H2 are mathematical codes for intermediate results, H1 represents the calculation result after the layer normalization operation in Transformer, and H2 represents the output result after the feedforward neural network calculation.
[0038] In step S3, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the fusion representation of the global information and historical information of the instrument data is completed, including:
[0039] For the relationship matrix obtained in step S2, each row represents the feature vector of each instrument node: H = {H1,…,H n}; Perform a linear transformation on the feature vector of each node to obtain a new feature vector:
[0040]
[0041] Among them, H represents the set of all n eigenvectors, represents the result of linear transformation of the ith eigenvector, represents the set of new feature vectors obtained after the i transformation, W is the linear transformation matrix; the graph attention network GAT is introduced, based on the feature vector of node i, j and τ to obtain the representation output O that integrates the global information i :
[0042]
[0043] Among them, LeakyRelu(·) represents an activation function, represents the attention weight in the GAT calculation process, K represents the number of GAT layers in the graph attention network, and α ij represents the attention weight between the oth node and the jth node, KeakyRelu() represents a well-known activation function in the field of deep learning, and exp() represents an exponential calculation operation. represents the feature vector of a node other than i and j, σ represents the nonlinear mapping function in the neural network, N i represents all adjacent nodes of the i-th node, W k represents the fusion weight of the kth node, τ j Represents the output of the j-th node Transformer.
[0044] In step S4, anomaly detection includes:
[0045] The output vector O of the graph attention network GAT i After linear transformation (W y x+b y ) and softmax layer output:
[0046] y=softmax(W y O i +b y )
[0047] Among them, y represents the predicted output of softmax, that is, the probability of whether the node is abnormal, softmax() represents the softmax calculation operation in numerical calculation, and W y represents the linear model weight of the anomaly detection module, b y represents the linear model bias;
[0048] And the cross entropy loss function is used to calculate the model loss:
[0049]
[0050] Where N represents the sample size, y i Represents the anomaly detection probability of the model output, y i ' represents the actual situation, 0 represents normal, and 1 represents abnormal.
[0051] Another object of the present invention is to provide a multi-source heterogeneous instrument data fusion and anomaly detection system, which implements the multi-source heterogeneous instrument data fusion and anomaly detection method, and the system includes:
[0052] A heterogeneous instrument data network diagram definition module is used to represent the data of different instruments as a graph structure by defining a heterogeneous instrument data network diagram;
[0053] The instrument node data similarity learning module is used to use graph convolutional neural networks to learn the low-dimensional embedding vector of each node from neighboring nodes, capture the implicit dependencies between nodes, and achieve deep fusion of multi-source heterogeneous data;
[0054] The representation learning module for instrument time series data is used to perform representation learning on instrument time series data using the Transformer model. By learning the context information of historical data, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the fusion representation of the global information and historical information of the instrument data is completed;
[0055] The anomaly detection module is used to calculate the probability of anomaly detection by transforming the output vector of the graph attention network and using the cross entropy loss function for model training.
[0056] Furthermore, the multi-source heterogeneous instrument data fusion and anomaly detection system is applied to data fusion and anomaly detection of different intelligent instruments in the industrial production field.
[0057] Combining all the above technical solutions, the beneficial effects of the present invention are as follows: the present invention is used for abnormal data detection of instruments and meters, can effectively capture the complex relationship between data, and has strong expression ability. The present invention defines a heterogeneous instrument data network diagram, represents the data of different instruments and meters as a graph structure, and uses graph convolutional neural network technology to learn the low-dimensional embedding vector of each node from neighboring nodes, capture the implicit dependency relationship between nodes, and realize the deep fusion of multi-source heterogeneous data.
[0058] Representation learning of instrument time series data: This paper adopts the Transformer model to perform representation learning on instrument time series data. By learning the context information of historical data, using the multi-head attention mechanism to capture the important features in the time series data, and combining the output of the graph neural network, the fusion representation of the global information and historical information of the instrument data is realized.
[0059] The present invention provides an anomaly detection method based on graph neural network data fusion: the probability of anomaly detection is calculated by transforming the output vector of the graph attention network, and the cross entropy loss function is used for model training. The method can automatically process multi-source heterogeneous data, reduce dependence on manual operation, and improve the accuracy and efficiency of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure;
[0061] Figure 1 It is a flow chart of a method for multi-source heterogeneous instrument data fusion and anomaly detection provided by an embodiment of the present invention;
[0062] Figure 2 It is a schematic diagram of a method for multi-source heterogeneous instrument data fusion and anomaly detection provided by an embodiment of the present invention;
[0063] Figure 3 Schematic diagram of a multi-source heterogeneous instrument data fusion and anomaly detection system provided by an embodiment of the present invention;
[0064] In the figure: 1. Heterogeneous instrument data network diagram definition module; 2. Instrument node data similarity learning module; 3. Instrument time series data representation learning module; 4. Anomaly detection module. DETAILED DESCRIPTION
[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.
[0066] Technical terms related to the present invention:
[0067] Graph Neural Network (GNN): is a deep learning model for processing graph structured data. It performs tasks such as node classification, graph classification, and link prediction by learning the relationships between nodes and the topological structure of the graph.
[0068] Graph Convolutional Network (GCN) is a deep learning model for processing graph-structured data. It extracts features and makes predictions by applying convolution operations on graph nodes.
[0069] Graph Attention Network (GAT) is a deep learning model that introduces an attention mechanism to dynamically adjust the weights between nodes, thereby more effectively processing graph structured data.
[0070] Non-negative Matrix Factorization (NMF) is a dimensionality reduction technique that decomposes a non-negative matrix into the product of two non-negative matrices for feature extraction and data representation.
[0071] Anomaly Detection: refers to the process of identifying abnormal data points or patterns in a data set that are significantly different from the majority of the data. These abnormal data points may represent errors, fraud, failures, or other special situations that require attention. Anomaly detection is widely used in various fields, such as network security, financial fraud detection, equipment failure prediction, and medical diagnosis. Its purpose is to prevent the occurrence or spread of potential problems by identifying abnormal situations and taking timely measures. Common anomaly detection methods include statistical analysis, machine learning, and deep learning.
[0072] The innovation of this invention is that by defining a heterogeneous instrument data network graph, using a graph convolutional neural network (GCN) to achieve deep data fusion, and using a Transformer model to represent and learn time series data, and finally performing anomaly detection through a graph attention network. This method can automatically process multi-source heterogeneous data, significantly improve data processing efficiency and anomaly detection accuracy, enhance data fusion and feature extraction capabilities, improve the scalability and adaptability of the system, and realize intelligent data management.
[0073] Embodiment 1: The method for multi-source heterogeneous instrument data fusion and anomaly detection provided by the embodiment of the present invention includes:
[0074] S1, by defining a heterogeneous instrument data network graph, the data of different instruments are represented as a graph structure;
[0075] S2, instrument node data similarity learning: using graph convolutional neural networks, learn the low-dimensional embedding vector of each node from neighboring nodes, capture the implicit dependencies between nodes, and achieve deep fusion of multi-source heterogeneous data;
[0076] S3, representation learning of instrument time series data: The Transformer model is used to learn the representation of instrument time series data. By learning the context information of historical data, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the global information and historical information of the instrument data are integrated.
[0077] S4, anomaly detection: The probability of anomaly detection is calculated by transforming the output vector of the graph attention network, and the cross entropy loss function is used to perform end-to-end training on the graph convolutional neural network, Transformer model and graph attention network model.
[0078] In step S1, defining a heterogeneous instrument data network diagram includes:
[0079] Define a heterogeneous instrument data network as a heterogeneous instrument data network graph G = (V, E, A, X), where V ∈ R n represents the instrument node space of graph G (R is the real number domain), n represents the number of instrument nodes; E represents the edge of graph G; A∈R n*n is the adjacency matrix of graph G; for two instrument nodes v i ,v j ∈V, if edge e ij ∈E (i.e., there is a physical connection or upstream and downstream relationship between the two instrument nodes), then A ij =1, otherwise A ij =0, A ij is the value of the i-th row and j-th column of the adjacency matrix A; X∈R n*m Represents the attribute information of the instrument node, and m is the dimension of the attribute information.
[0080] In step S2, the instrument node data similarity learning specifically includes:
[0081] After obtaining the topological relationship modeling diagram between the instruments, that is, the heterogeneous instrument data network diagram, the Graph Convolutional Network (GCN) is used to learn the low-dimensional embedding vector of each instrument node from the neighboring instrument nodes through convolution and aggregation operators. The convolution operations of each layer are as follows:
[0082]
[0083] Among them, H i It represents the output of the i-th layer network of GCN, and is also the network input of the i+1th layer. Where I represents the diagonal bias matrix; The diagonal bias matrix represented by W i represents the set of transformation parameters of the i-th layer network; δ represents the activation function.
[0084] In the i-1th layer, the non-negative matrix factorization method (NNMF) is used to factorize the H output of each layer of the previous GCN network. i Decompose into 2 low-dimensional matrices:
[0085] H i =U i V i ,sT i ≥0,Vi ≥0;
[0086] in L i represents the output dimension of the i-th layer, d is the output dimension of the NNMF method, then U i Indicates H i A lower dimensional representation vector. The low-dimensional vectors of each layer are concatenated and mapped to n dimensions through a single hidden layer perceptron (Multi-Layer Perceptron, MLP). The final output can be expressed as:
[0087] H i ′=MLP(‖U0‖‖U1‖…‖U i-1 ‖);
[0088] Where ‖·‖ represents feature concatenation.
[0089] In step S3, the representation learning of instrument time series data specifically includes:
[0090] It is used to learn time series representation for different instruments. Based on the historical data of a previous time window, the Transformer attention model is used to learn the context information of the instrument data. The calculation process is as follows:
[0091] Q,K,V=embedding(X);
[0092]
[0093] head i =Attention(QW i Q ,KW i K ,VW i V );
[0094] Multihead(Q,K,V)=cibcat(head1,…,head n )W o ;
[0095] in All are Transformer model parameter vectors, embedding represents vector mapping operation, and softmax represents normalized exponential function. After obtaining the multi-head attention vector representation (Multi-head Attention), we can further obtain the Transformer output τ:
[0096] H1=LN(Multihead(Q,K,V)+X);
[0097] H2=FFN(H1);
[0098] τ=LN(H1+H2);
[0099] Among them, LN(·) represents the layer normalization method (Layer Normalization), and FFN(·) represents the feed forward neural network layer (Feed Forward Layer). Transformer is used to obtain the information representation of each instrument combined with the historical data time series, and the graph attention network (GAT) is used to capture the implicit dependency between the instrument node data in combination with the instrument node similarity information matrix obtained in step S2, thereby realizing the data representation learning that integrates global and historical information.
[0100] The specific process is as follows: For the relationship matrix obtained in step S2, each row can be expressed as the characteristic vector of each instrument node: H = {H1,…,H n}(For simplicity, H i ′ is denoted as H). Perform a linear transformation on the feature vector of each node to obtain a new feature vector:
[0101]
[0102] Where W is the linear transformation matrix. Next, we introduce a graph attention network GAT based on the feature vector of node i, j. and τ to obtain the representation output O that integrates the global information i :
[0103]
[0104] Where LeakyRelu(·) represents an activation function, represents the attention weight in the GAT calculation process, and K represents the number of GAT layers in the graph attention network (the calculation process can be described in detail using existing technology P.,Cucurull,G.,Casanova,A.,Romero,A.,Liò,P.,&Bengio,Y.(2018,February).Graph AttentionNetworks.In International Conference on Learning Representations).
[0105] In step S4, anomaly detection includes:
[0106] In order to achieve anomaly detection, the output vector O of the graph attention network GAT can be i After a simple linear transformation (W y x+b y ) and softmax layer output:
[0107] y=softmax(W y O i +b y );
[0108] And the cross entropy loss function is used to calculate the model loss:
[0109]
[0110] Where N represents the sample size, y i Represents the anomaly detection probability of the model output, y i ' represents the true situation (0 is normal, 1 is abnormal).
[0111] In an embodiment of the present invention, Figure 2 It is the method and principle of multi-source heterogeneous instrument data fusion and anomaly detection.
[0112] As another embodiment of the present invention, in step S3, the Transformer model is used to aggregate historical data information for each instrument, and the graph attention network is used as a model for aggregating global information. However, the present invention does not impose specific constraints on time series fusion and graph neural network models, and can be replaced by various algorithms or models. For example, historical data fusion can directly splice historical instrument data into multi-dimensional features, or through other time series modeling methods; the graph neural network can also be replaced by other advanced models (such as GraphSAGE, etc.). It can be expanded or refined according to the constraints on computing resources, memory usage, and accuracy requirements of actual applications.
[0113] The present invention only analyzes the task of anomaly detection, but the actual multi-source data fusion framework can be extended to other downstream tasks, such as numerical prediction, node classification, etc., by simply modifying the loss function in step S4.
[0114] It can be seen from the above embodiments that the present invention improves data processing efficiency and accuracy: by introducing machine learning technologies such as graph neural networks (GNN), the present invention can automatically process multi-source heterogeneous data and reduce reliance on manual operations. This not only reduces manpower and material costs, but also avoids errors and inaccuracies that may be caused by manual detection.
[0115] The present invention enhances the data fusion and feature extraction capabilities: traditional methods often find it difficult to fully utilize the correlation information between data when processing multi-source heterogeneous data. The present invention achieves deep data fusion and feature extraction through graph convolutional neural networks (GCN), Transformer models, and graph attention networks (GAT). GCN can learn the low-dimensional embedding vector of each node from neighboring nodes, the Transformer model is good at processing time series data, and GAT further captures the implicit dependencies between nodes. This multi-level, multi-angle data fusion and feature extraction method enables the system to understand and utilize data more comprehensively, improving the effect of anomaly detection.
[0116] The present invention improves the scalability and adaptability of the system: The technical solution of the present invention has strong scalability and adaptability. First, the introduction of graph neural networks and Transformer models enables the system to process large-scale, real-time heterogeneous data. Secondly, the design of the system allows different time series modeling methods and graph neural network models to be replaced to adapt to different application scenarios and computing resource constraints. In addition, the system is not limited to anomaly detection tasks, but can also be extended to other downstream tasks such as numerical prediction and node classification by modifying the loss function. This flexibility enables the system to adapt to changing business needs and technological developments.
[0117] The present invention can further realize intelligent and visualized data management: through the topological relationship modeling of the graph neural network and the global information fusion of the graph attention network, the present invention can realize the intelligent management of complex logical data. Based on the intermediate results such as the instrument node data similarity vector and representation vector proposed by the present invention, the topological relationship between instruments can be further managed and viewed in combination with the data visualization method, making the data context relationship clearer and more visualized. This intelligent data management method not only improves the efficiency of data analysis, but also provides strong support for "tracing the source" data verification.
[0118] Embodiment 2, as Figure 3 As shown, the multi-source heterogeneous instrument data fusion and anomaly detection system provided by the embodiment of the present invention includes:
[0119] A heterogeneous instrument data network diagram definition module 1 is used to represent the data of different instruments as a graph structure by defining a heterogeneous instrument data network diagram;
[0120] Instrument node data similarity learning module 2 is used to use graph convolutional neural networks to learn the low-dimensional embedding vector of each node from neighboring nodes, capture the implicit dependencies between nodes, and achieve deep fusion of multi-source heterogeneous data;
[0121] The representation learning module 3 of instrument time series data is used to perform representation learning on instrument time series data using the Transformer model. By learning the context information of historical data, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the fusion representation of the global information and historical information of the instrument data is completed;
[0122] The anomaly detection module 4 is used to calculate the probability of anomaly detection by transforming the output vector of the graph attention network and using the cross entropy loss function for model training.
[0123] The present invention improves production efficiency and reduces costs. By automatically processing multi-source heterogeneous data, the reliance on manual operations is reduced, and the data processing efficiency and accuracy are significantly improved; at the same time, the reliance on data experts and business experts is reduced, and the manpower and material costs are reduced.
[0124] Instrumentation product and service upgrades and new market development. This technical solution can be integrated into existing industrial IoT platforms and data analysis systems related to instrumentation to enhance the added value of products and services; in addition, this technical solution is not only applicable to industrial production, but can also be extended to other fields that require processing of multi-source heterogeneous data, such as smart grids, smart cities, medical health, etc., to open up new market opportunities.
[0125] Multi-source heterogeneous data fusion in instrumentation: When processing multi-source heterogeneous data, existing technologies often find it difficult to fully utilize the correlation information between data. This invention achieves deep fusion of multi-source heterogeneous data through models such as graph neural networks and Transformer, filling this technical gap in the instrumentation industry.
[0126] With the rapid development of Industry 4.0 and Internet of Things technologies, the types and number of instruments used in industrial production processes have increased dramatically. These instruments usually come from different manufacturers and have different communication protocols and data formats, forming multi-source heterogeneous data. How to effectively fuse these multi-source heterogeneous data and extract valuable information is an important challenge in the current field of industrial data analysis. This technical solution overcomes the problems of missing important features and poor scalability caused by relying on manual operations in traditional methods. Through models such as graph neural networks and Transformer, the fusion of multi-source data significantly improves the accuracy and efficiency of anomaly detection, alleviating this problem.
[0127] Dependence on manual operations: Traditional methods rely on data experts and business experts to perform data preprocessing and anomaly detection, which is highly subjective and uncertain. This invention reduces the reliance on manual operations by automatically processing multi-source heterogeneous data, thus overcoming this technical bias.
[0128] Limitations of feature engineering: Traditional methods rely on expert knowledge for feature engineering, which easily misses important features and is difficult to automate batch processing. This paper overcomes this technical bias by achieving deep data fusion and feature extraction through graph neural networks and Transformer models.
[0129] To further illustrate the effects of the embodiments of the present invention, the following experiment was conducted.
[0130] In recent years, graph neural networks have achieved remarkable results in many application scenarios such as social network analysis, knowledge graphs, and bioinformatics. From a theoretical perspective, this solution has the following advantages over traditional methods when fusing multi-source instrument data, as shown in Table 1;
[0131] Table 1 Comparison between the present invention and the traditional method in fusing multi-source instrument data
[0132]
[0133]
[0134] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.
Claims
1. A method for multi-source heterogeneous instrument data fusion and anomaly detection, characterized in that: The method includes: S1, by defining a heterogeneous instrument data network graph, the data of different instruments are represented as a graph structure; S2, instrument node data similarity learning: using graph convolutional neural networks, learn the low-dimensional embedding vector of each node from neighboring nodes, capture the implicit dependencies between nodes, and achieve deep fusion of multi-source heterogeneous data; S3, representation learning of instrument time series data: The Transformer model is used to learn the representation of instrument time series data. By learning the context information of historical data, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the global information and historical information of the instrument data are integrated. S4, anomaly detection: The probability of anomaly detection is calculated by transforming the output vector of the graph attention network, and the cross entropy loss function is used for end-to-end training of the graph convolutional neural network, Transformer model and graph attention network model.
2. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 1 is characterized in that: In step S1, a heterogeneous instrument data network diagram is defined, including: Define a heterogeneous instrument data network as a heterogeneous instrument data network graph G = (V, E, A, X), where V ∈ R n represents the instrument node space of graph G, R is the real number field, n represents the number of instrument nodes; E represents the edge of graph G; A∈R n*n is the adjacency matrix of graph G; for two instrument nodes v i , v j ∈V, if edge e ij ∈E, then A ij =1, otherwise A ij =0, A ij is the value of the i-th row and j-th column of the adjacency matrix A; X∈R n * m Represents the attribute information of the instrument node, and m is the dimension of the attribute information.
3. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 2 is characterized in that: In step S2, instrument node data similarity learning includes: After obtaining the heterogeneous instrument data network graph, the graph convolutional neural network (GCN) is used to learn the low-dimensional embedding vector of each instrument node from neighboring instrument nodes through convolution and aggregation operators. In the i-1 layer, the non-negative matrix factorization method NNMF is used to convert the H output of each layer of the graph convolutional neural network GCN network i Decomposed into two low-dimensional matrices; among them, H i It represents the output of the i-th layer network of GCN, and is also the network input of the i+1th layer; The low-dimensional vectors of each layer are concatenated and mapped to n dimensions through a single hidden layer perceptron MLP and output.
4. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 3 is characterized in that: In the graph convolutional neural network GCN, the convolution operations of each layer are as follows: Among them, H i+1 represents the output of the i+1th layer network of GCN, δ represents the activation function, The diagonal bias matrix represented by Represents the value of the i-th row and j-th column of the diagonal matrix; represents the sum of the adjacency matrix of graph G and the diagonal bias matrix, A represents the adjacency matrix of graph G, and I represents the diagonal bias matrix; W i Represents the set of transformation parameters of the i-th layer network.
5. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 4 is characterized in that: In the i-1 layer, the non-negative matrix factorization method NNMF is used to convert the H output of each layer of the GCN network i The expression decomposed into two low-dimensional matrices is: H i =U i V i ,s.t.U i ≥0,V i ≥0 in, U i ∈R n*d , L i represents the output dimension of the i-th layer, d is the output dimension of the NNMF method, and U i Indicates H i A lower dimensional representation vector, V i Indicates H i Another part of matrix factorization; Mapped to n dimensions through a single hidden layer perceptron MLP, the final output is expressed as: H i ′=MLP(||U0||||U1||...||U i-1 ||) Among them, ||·|| represents feature concatenation, H i ′ represents the final output vector of the GCN network, MLP() represents the multi-layer perceptron model, U0 represents the low-dimensional representation vector of the 0th layer after matrix decomposition, U1 represents the low-dimensional representation vector of the 1st layer after matrix decomposition, and U i-1 Represents the low-dimensional representation vector of the i-1th layer after matrix decomposition.
6. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 1 is characterized in that: In step S3, the representation learning of instrument time series data includes: Based on the historical data of a certain time window, the Transformer attention model is used to learn the context information of the instrument data; the calculation process is as follows: Q, K, V = embedding(X) head i =Attention(QW i Q ,KW i K ,VW i V ) Multihead(Q,K,V)=concat(head1,...,head n )W o in, are all Transformer model parameter vectors; after obtaining the multi-head attention vector representation, the Transformer output τ is further obtained: Q represents the query matrix of the Transformer model, K represents the key matrix of the Transformer model, V represents the value matrix of the Transformer model, embedding(X) represents the mapping model of the instrument node feature X, Attention() represents the attention calculation operation of the Transformer model, softmax() represents the softmax mathematical calculation operation, T represents the matrix transpose operation, d k Represents the dimension of the K matrix in Transformer, head i represents the i-th output head of the Transformer model, W i Represents the W matrix of the i-th layer of the Transformer model, Multihead() represents the calculation operation of multiple output heads of the Transformer model, concat() represents the vector concatenation operation, head1 represents the first output head of the Transformer model, head n represents the nth output head of the Transformer model, W o Represents the matrix representation of the output layer of the Transformer model; H1=LN(Mutlihead(Q,K,V)+X) H2=FFN(H1) τ=LN(H1+H2) Among them, LN(·) represents the layer normalization method, FFN(·) represents the feedforward neural network layer, H1 and H2 are mathematical codes for intermediate results, H1 represents the calculation result after the layer normalization operation in Transformer, and H2 represents the output result after the feedforward neural network calculation.
7. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 6 is characterized in that: In step S3, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the fusion representation of the global information and historical information of the instrument data is completed, including: For the relationship matrix obtained in step S2, each row represents the feature vector of each instrument node: H = {H1, ..., H n }; Perform a linear transformation on the feature vector of each node to obtain a new feature vector: Among them, H represents the set of all n eigenvectors, represents the result of linear transformation of the ith eigenvector, represents the set of new feature vectors obtained after the i transformation, W is the linear transformation matrix; the graph attention network GAT is introduced, based on the feature vectors of nodes i and j and τ to obtain the representation output O that integrates the global information i : Among them, LeakyRelu(·) represents an activation function, represents the attention weight in the GAT calculation process, K represents the number of GAT layers in the graph attention network, and α ij represents the attention weight between the i-th node and the j-th node, LeakyRelu() represents a well-known activation function in the field of deep learning, and exp() represents an exponential calculation operation. represents the feature vector of a node other than i and j, σ represents the nonlinear mapping function in the neural network, N i represents all adjacent nodes of the i-th node, W k represents the fusion weight of the kth node, τ j Represents the output of the j-th node Transformer.
8. The method for multi-source heterogeneous instrument data fusion and anomaly detection according to claim 1 is characterized in that: In step S4, anomaly detection includes: The output vector O of the graph attention network GAT i After linear transformation (W y x+b y ) and softmax layer output: y =softmax(W y O i +b y ) Among them, y represents the predicted output of softmax, that is, the probability of whether the node is abnormal, softmax() represents the softmax calculation operation in numerical calculation, and W y represents the linear model weight of the anomaly detection module, b y represents the linear model bias; And the cross entropy loss function is used to calculate the model loss: Where N represents the sample size, y i Represents the anomaly detection probability of the model output, y i ' represents the actual situation, 0 represents normal, and 1 represents abnormal.
9. A multi-source heterogeneous instrument data fusion and anomaly detection system, characterized in that: The system implements the method for multi-source heterogeneous instrument data fusion and anomaly detection as claimed in any one of claims 1 to 8, and the system comprises: A heterogeneous instrument data network diagram definition module (1), used for representing data of different instruments as a graph structure by defining a heterogeneous instrument data network diagram; The instrument node data similarity learning module (2) is used to use a graph convolutional neural network to learn a low-dimensional embedding vector of each node from its neighboring nodes, capture the implicit dependency between nodes, and achieve deep fusion of multi-source heterogeneous data; The representation learning module (3) of instrument time series data is used to perform representation learning on instrument time series data using the Transformer model. By learning the context information of historical data, the multi-head attention mechanism is used to capture the important features in the time series data, and combined with the output of the graph attention network, the fusion representation of the global information and historical information of the instrument data is completed; The anomaly detection module (4) is used to calculate the probability of anomaly detection by transforming the output vector of the graph attention network and adopting the cross entropy loss function to train the model.
10. The multi-source heterogeneous instrument data fusion and anomaly detection system according to claim 9, characterized in that: The multi-source heterogeneous instrument data fusion and anomaly detection system is applied to data fusion and anomaly detection of different intelligent instruments in the industrial production field.
Citation Information
Cited By
Material cladding layer performance prediction method and system based on big data
CN119517221A
Agricultural product quality traceability anomaly detection method
CN120410342A
Data joint debugging method and system based on heterogeneous monitoring system
CN120803853A
