Anomaly detection method for industrial control systems based on Graphomer

Through the Graphomer exception detection method and combined with the federal learning framework, the cause of failure of the industrial control system is identified, the problem of system paralysis is solved, efficient abnormal detection and fault tracing is achieved, and economic losses are reduced.

CN115034304BActive Publication Date: 2024-12-03GUANGDONG POLYTECHNIC NORMAL UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210650680.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-09
Publication Date
2024-12-03
Estimated Expiration
2042-06-09

AI Technical Summary

Technical Problem

Industrial control systems are prone to paralysis when facing network intrusions and failures, resulting in economic losses. It is difficult for existing technology to effectively identify the causes of failures and deal with them in a timely manner.

Method used

Using the Graphomer-based industrial control system anomaly detection method, through system initialization, model training, parameter aggregation, data behavior association rules construction, multimodal association graph construction and fault traceability, Graphomer's point prediction function is used to identify the source of the abnormality, combined with federated learning The framework protects data security.

Benefits of technology

Effectively detect and explain industrial system abnormalities, quickly identify the causes of failures, reduce economic losses, and build an abnormality detection model with high safety and high accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115034304B_ABST
    Figure CN115034304B_ABST
Patent Text Reader

Abstract

The present invention discloses a Graphomer-based industrial control system anomaly detection method. In the initial stage, the cloud server gives initialization parameters #imgabs0# based on the anomaly detection model, including anomaly detection model parameters q 0 , and some parameters that are helpful for federated learning aggregation models. This Graphomer-based industrial control system anomaly detection method uses concurrent association rules to build association graphs between various data, more effectively represents the association relationship between data, and establishes an industrial system anomaly detection model based on Graphomer; this model can effectively detect anomalies in industrial systems, explain anomalies, and find the source of anomalies; by developing a federated learning framework, in addition, through this framework, the parameters of the same model trained by different data sources can be referred to, which can well protect the security and privacy of industrial data during the training process, thereby constructing a highly secure and accurate anomaly detection model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of industrial production, and in particular to a graphomer-based industrial control system anomaly detection method. Background Art

[0002] The Industrial Internet has achieved a close integration of the industrial field and the Internet, and represents a promising paradigm for industrial operation efficiency and innovation. Industrial control systems connect industrial elements such as equipment, data, and personnel by combining machine learning with network technology, and allow the application of analytical tools and network technology to manage industrial operations and provide new value-added services. They play an extremely important role in production applications. As working time goes by, key equipment in the production field is prone to various failures, such as aging of parts, external environmental influences, and improper human operation. These failures may cause certain mechanical properties of the equipment to deteriorate or even paralyze, causing unnecessary production suspensions and economic losses.

[0003] Under the background of industrial Internet, industrial control systems are transforming and upgrading towards intelligence, which has greatly improved production efficiency and economic benefits. However, while bringing benefits, it also increases the risk of network intrusion attacks. Moreover, as the operation time goes by, industrial control equipment is prone to various failures. These intrusions, failures and other anomalies may cause the paralysis of industrial control systems and cause huge economic losses. Therefore, it is necessary to design an effective anomaly detection model for industrial control systems. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides an industrial control system anomaly detection method based on Graphomer, which has the advantages of being able to identify the cause of system failures through fault diagnosis technology, thereby promptly handling the failures and reducing losses. It solves the problem that network intrusions, failures and other anomalies may cause the paralysis of industrial control systems and cause huge economic losses.

[0006] (II) Technical solution

[0007] In order to achieve the above-mentioned purpose of being able to identify the cause of system failure through fault diagnosis technology, thereby promptly handling the failure and reducing losses, the present invention provides the following technical solution: a method for detecting abnormalities in an industrial control system based on Graphomer, characterized in that it includes the following steps:

[0008] S1. System framework workflow

[0009] S1.1. System initialization

[0010] In the initial stage, the cloud server gives initialization parameters based on the anomaly detection model. Including the anomaly detection model parameters q 0 , and some parameters that contribute to the federated learning aggregation model, such as the learning rate η, the decay exponent ρ 1 ,ρ 2 ∈[0,1), a small constant ζ for stability, the loss function Batch size B, contribution rate α k , the number of communication rounds R, the total size of local data N k ;

[0011] S1.2. Ontology model training

[0012] Each industrial agent A k Receive initialization parameters from the cloud server and then use local private data D k Train the anomaly detection model; the detailed training process is given in Algorithm 2; since the industrial agent belongs to the edge server, it has sufficient computing power to support local model training;

[0013] S1.3. Model parameter aggregation

[0014] According to the contribution rate of all industrial agents, the cloud server aggregates all parameters and then sends the aggregated parameters to all industrial agents;

[0015] S1.4. Local model update

[0016] The local agent receives the aggregation parameters of the rth round, uses the same offline local multimodal data to update the model, and re-uploads the updated model parameters; after R rounds of model parameter interaction, the final multimodal anomaly detection model is determined.

[0017] S2. Construction of data behavior association rules

[0018] Use sliding windows to slice the collected multimodal data, and build association rule tables between data behaviors based on different combinations of data behaviors under different data slices.

[0019] S3. Construction of multimodal association graph

[0020] Select one set of sliced ​​data and construct the association graph between the data according to the association rule table as the input of the Graphomer model;

[0021] S4. Graphomer-based point prediction

[0022] In order to detect point anomalies, we use the prediction function of Graphomer to predict the expected behavior at each time point, thereby identifying anomalies and finding the source of anomalies. Graphomer

[42] is a modification of Transformer that adds graph structure information encoding to effectively use the graph structure information. There are three types of graph structure information encoding, namely center encoding, space encoding, and edge encoding.

[0023] S5. Fault tracing

[0024] Fault tracing can be viewed as a binary classification problem, that is, determining whether different nodes are abnormal at the current moment. By converting the predicted values ​​into anomaly scores, nodes can be labeled as normal or abnormal, and a convergence function can be constructed based on the predicted labels of the nodes to train the model.

[0025] Preferably, in step S1, the system framework workflow mainly includes two types of entities, namely, cloud server C and industrial agent A k ,k∈K, connect the industrial agents of multiple industrial control systems to the same cloud server and jointly build an anomaly detection model.

[0026] Preferably, in step S1, a comprehensive anomaly detection model is established. During initialization, initialization parameters are created. Send it to each industrial agent; in the rth round of training, combine the model parameters trained locally in each industrial agent Then it is sent to various industrial agents.

[0027] Preferably, in step S1, R rounds (optimal number of rounds obtained from experience) of interaction are performed between the cloud server and each industrial agent, and the final anomaly detection model is obtained through parameter training, aggregation, and interaction.

[0028] Preferably, in step S2, industrial agent A k The local data in is defined as Use a sliding window with a length of B and a step size of l to intercept data D k , obtain X data slices of length B, where Among them, floor(·) represents the rounding function, D X Represents the total data set after sliding window processing.

[0029] Preferably, in step S3, any slice data D is selected from the data slices. x ,D x ∈D X As a training batch, in the current data batch In , we take the first B-1 length as the input of the anomaly detection model:

[0030] x (B) =[s 1 ,s 2 ,…,s (B-1) ].

[0031] Preferably, in step S4, in Graphormer, central coding is used to represent the importance of nodes in the graph, and in the input layer, node features are converted into and the learnable degree vector Addition;

[0032]

[0033] Preferably, in step S4, in Graphormer, spatial coding is used to represent the global position relationship of nodes, not only considering the local relationship between the current node and the adjacent nodes, but also calculating the minimum path between the nodes. To represent the node v i ,v j If the two nodes are not related, A i,j Expressed as the i-th row and j-th column element in the Query-Key product matrix A, we have:

[0034]

[0035] in is a through The weight matrix can be learned and shared among all layers; h i is the output of the center encoding.

[0036] Preferably, in step S4, in order to better encode edge features into the attention layer, edge encoding is used to represent the relevance of nodes in Graphormer; first, two nodes v are found i ,v j The shortest path between SP i,j =(e 1 ,e 2 ,…,e M ) and calculate the average of the edge feature dot products and the learnable embedding path.

[0037] (III) Beneficial effects

[0038] Compared with the prior art, the present invention provides an industrial control system anomaly detection method based on Graphomer, which has the following beneficial effects:

[0039] 1. The Graphomer-based industrial control system anomaly detection method constructs an association graph between various data by utilizing concurrent association rules, more effectively represents the association relationship between data, and establishes an industrial system anomaly detection model based on Graphomer; the model can effectively detect anomalies in industrial systems, explain anomalies, and find the source of anomalies.

[0040] 2. The Graphomer-based industrial control system anomaly detection method develops a federated learning framework. Each industrial user uses data resources to train the model. In addition, this framework can refer to the parameters of the same model trained with different data sources, which can well protect the security and privacy of industrial data during the training process, thereby constructing a highly secure and accurate anomaly detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a diagram of an anomaly detection system based on federated learning;

[0042] Figure 2 It is a flow chart of the fault diagnosis framework. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Embodiment 1:

[0045] A graphomer-based industrial control system anomaly detection method comprises the following steps:

[0046] S1. System framework workflow

[0047] S1.1. System initialization

[0048] In the initial stage, the cloud server gives initialization parameters based on the anomaly detection model. Including the anomaly detection model parameters q 0 , and some parameters that contribute to the federated learning aggregation model, such as the learning rate η, the decay exponent ρ 1 ,ρ 2 ∈[0,1), a small constant ζ for stability, the loss function Batch size B, contribution rate α k , the number of communication rounds R, the total size of local data N k ;

[0049] S1.2. Ontology model training

[0050] Each industrial agent A k Receive initialization parameters from the cloud server and then use local private data D k Train the anomaly detection model. The detailed training process is given in Algorithm 2. Since the industrial agent belongs to the edge server, it has sufficient computing power to support local model training.

[0051] S1.3. Model parameter aggregation

[0052] According to the contribution rate of all industrial agents, the cloud server aggregates all parameters and then sends the aggregated parameters to all industrial agents;

[0053] S1.4. Local model update

[0054] The local agent receives the aggregated parameters of the rth round, uses the same offline local multimodal data to update the model, and re-uploads the updated model parameters. After R rounds of model parameter interaction, the final multimodal anomaly detection model is determined.

[0055] The system framework workflow mainly includes two types of entities, namely cloud server C and industrial agent A k ,k∈K, connect the industrial agents of multiple industrial control systems to the same cloud server and jointly build an anomaly detection model.

[0056] Responsible for building a comprehensive anomaly detection model. During initialization, create initialization parameters Send it to each industrial agent; in the rth round of training, combine the model parameters trained locally in each industrial agent Then it is sent to various industrial agents.

[0057] The final anomaly detection model is obtained through R rounds (the optimal number of rounds obtained from experience) of interaction between the cloud server and each industrial agent, and after parameter training, aggregation, and interaction.

[0058] See Algorithm 1 below for details:

[0059]

[0060] S2. Construction of data behavior association rules

[0061] Use sliding windows to slice the collected multimodal data, and build association rule tables between data behaviors based on different combinations of data behaviors under different data slices.

[0062] Industrial Agent A k The local data in is defined as Use a sliding window with a length of B and a step size of l to intercept data D k , obtain X data slices of length B, where Among them, floor(·) represents the rounding function, D X Represents the total data set after sliding window processing.

[0063] S3. Construction of multimodal association graph

[0064] Select one set of slice data, build the association graph between the data according to the association rule table as the input of the Graphomer model; select any slice data D from the data slices x ,D x ∈D X As a training batch, in the current data batch In , we take the first B-1 length as the input of the anomaly detection model:

[0065] x (B) =[s 1 ,s 2 ,…,s (B-1) ].

[0066] S4. Graphomer-based point prediction

[0067] In order to detect point anomalies, we use the prediction function of Graphomer to predict the expected behavior at each time point, thereby identifying anomalies and finding the source of anomalies. Graphomer

[42] is a modification of Transformer that adds graph structure information encoding to effectively use the graph structure information. There are three types of graph structure information encoding, namely center encoding, space encoding, and edge encoding.

[0068] In Graphormer, the central encoding is used to represent the importance of the node in the graph. In the input layer, the node features are and the learnable degree vector Addition;

[0069]

[0070] In Graphormer, spatial encoding is used to represent the global position relationship of nodes, not just the local relationship between the current node and the adjacent nodes. To represent the node v i ,v j If the two nodes are not related, A i,jExpressed as the i-th row and j-th column element in the Query-Key product matrix A, we have:

[0071]

[0072] in is a through The weight matrix can be learned and shared among all layers; h i is the output of the center encoding.

[0073] In order to better encode edge features into the attention layer, in Graphormer, edge encoding is used to represent the relevance of nodes; first find two nodes v i ,v j The shortest path between SP i,j =(e 1 ,e 2 ,…,e M ) and calculate the average of the edge feature dot products and the learnable embedding path.

[0074] S5. Fault tracing

[0075] Fault tracing can be viewed as a binary classification problem, that is, determining whether different nodes are abnormal at the current moment. By converting the predicted values ​​into anomaly scores, nodes can be labeled as normal or abnormal, and a convergence function can be constructed based on the predicted labels of the nodes to train the model.

[0076] Algorithm 2: Local anomaly detection algorithm training

[0077]

[0078]

[0079] Embodiment 2

[0080] A graphomer-based industrial control system anomaly detection method comprises the following steps:

[0081] S1. System framework workflow

[0082] S1.1. System initialization

[0083] In the initial stage, the cloud server gives initialization parameters based on the anomaly detection model. Including the anomaly detection model parameters q 0 , and some parameters that contribute to the federated learning aggregation model, such as the learning rate η, the decay exponent ρ 1 ,ρ 2 ∈[0,1), a small constant ζ for stability, the loss function Batch size B, contribution rate α k, the number of communication rounds R, the total size of local data N k ;

[0084] S1.2. Ontology model training

[0085] Each industrial agent A k Receive initialization parameters from the cloud server and then use local private data D k Train the anomaly detection model. The detailed training process is given in Algorithm 2. Since the industrial agent belongs to the edge server, it has sufficient computing power to support local model training.

[0086] S1.3. Model parameter aggregation

[0087] According to the contribution rate of all industrial agents, the cloud server aggregates all parameters and then sends the aggregated parameters to all industrial agents;

[0088] S1.4. Local model update

[0089] The local agent receives the aggregated parameters of the rth round, uses the same offline local multimodal data to update the model, and re-uploads the updated model parameters. After R rounds of model parameter interaction, the final multimodal anomaly detection model is determined.

[0090] The system framework workflow mainly includes two types of entities, namely cloud server C and industrial agent A k ,k∈K, connect the industrial agents of multiple industrial control systems to the same cloud server and jointly build an anomaly detection model.

[0091] Responsible for building a comprehensive anomaly detection model. During initialization, create initialization parameters Send it to each industrial agent; in the rth round of training, combine the model parameters trained locally in each industrial agent Then it is sent to various industrial agents.

[0092] The final anomaly detection model is obtained through R rounds (the optimal number of rounds obtained from experience) of interaction between the cloud server and each industrial agent, and after parameter training, aggregation, and interaction.

[0093] See Algorithm 1 below for details:

[0094] Algorithm 1: Federated Learning Framework for Anomaly Detection

[0095]

[0096]

[0097] S2. Construction of data behavior association rules

[0098] Use sliding windows to slice the collected multimodal data, and build association rule tables between data behaviors based on different combinations of data behaviors under different data slices.

[0099] Industrial Agent A k The local data in is defined as Use a sliding window with a length of B and a step size of l to intercept data D k , obtain X data slices of length B, where Among them, floor(·) represents the rounding function, D X Represents the total data set after sliding window processing.

[0100] Since the production behavior in the industrial control system is periodic, the behavior characteristics of different modal data are also periodic. The combination of behavior characteristics of different modal data is defined as the concurrent behavior rules of data behavior. There is a set of action behavior set r = {action 1 ,action 2 ,…}, where action i For the behavior action of one of the data in the current slice, the Apriori algorithm can be used to calculate the probability of concurrent association rules between different behaviors of different modal data, and then form a concurrent association rule table tab; the support calculation formula of the association rule r is as follows:

[0101]

[0102] Among them, |r| and |X| represent the number of occurrences of r and the total number of data sets after sliding window processing, respectively.

[0103] S3. Construction of multimodal association graph

[0104] Select one set of slice data, build the association graph between the data according to the association rule table as the input of the Graphomer model; select any slice data D from the data slices x ,D x ∈D X As a training batch, in the current data batch In , we take the first B-1 length as the input of the anomaly detection model:

[0105] x (B) =[s 1 ,s 2 ,…,s (B-1) ].

[0106] Our model needs to be based on x (B)The goal of the constructed association graph is to output the predicted values ​​of different modal types of data at time B, that is, s (B) Since the selected slice is fixed, the behavior pattern of the current modal data is fixed. Therefore, according to the concurrency association rule table, we can find The association probability between them is then used to construct an association graph G = {V, ε} with weight values, where The nodes representing the association graph are modal data, which are characterized by ε i,j is the edge of the association graph, indicating and The association, whose weight value w is equal to That is, the probability that data behavior actions appear simultaneously as association rules. The greater the association probability, the closer the connection, and the higher the association weight value of the edge in the association graph.

[0107] S4. Graphomer-based point prediction

[0108] In order to detect point anomalies, we use the prediction function of Graphomer to predict the expected behavior at each time point, thereby identifying anomalies and finding the source of anomalies. Graphomer

[42] is a modification of Transformer that adds graph structure information encoding to effectively use the graph structure information. There are three types of graph structure information encoding, namely center encoding, space encoding, and edge encoding.

[0109] In Graphormer, the central encoding is used to represent the importance of the node in the graph. In the input layer, the node features are and the learnable degree vector Addition;

[0110]

[0111] In Graphormer, spatial encoding is used to represent the global position relationship of nodes, not just the local relationship between the current node and the adjacent nodes. To represent the node v i ,v j If the two nodes are not related, A i,j Expressed as the i-th row and j-th column element in the Query-Key product matrix A, we have:

[0112]

[0113] in is a through The weight matrix can be learned and shared among all layers; h i is the output of the center encoding.

[0114] In order to better encode edge features into the attention layer, in Graphormer, edge encoding is used to represent the relevance of nodes; first find two nodes v i ,v j The shortest path between SP i,j =(e 1 ,e 2 ,…,e M ) and calculate the average of the edge feature dot products and the learnable embedding path.

[0115] Graphomer layer

[0116] The central code is used as the model input h (B) , consisting of three matrices Projected to the corresponding representations Q, K, V, respectively representing query, key, and value, the calculation formula is:

[0117] Q=h (B) W Q ,K=h (B) W K ,V=h (B) W V

[0118] After constructing the code, it is operated in the Graphormer model by Q, K, and V, such as Figure 2 As shown in part 3, the prediction of features between each node is achieved:

[0119]

[0120] in, is the hidden layer of the previous layer, is the hidden layer of the current layer output, LN(·) represents the normalization layer, which is used to standardize the optimization space and accelerate convergence, MHA(·) and FFN(·) are the multi-head self-attention layer and feed-forward network layer, respectively.

[0121] By traversing all nodes on the anomaly detection model, the features of all nodes are updated as Then, by comparing with the corresponding modal data v i Perform unit multiplication (expressed in °) to obtain the predicted value at time t

[0122]

[0123] in, One of the modal data vectors in the current training data batch.

[0124] Special Nodes

[0125] In order to better represent the overall information of the graph structure, a special node named [VNode] is added to Graphomer. In the AGGREGATE-COMBINE step, [VNode] is connected to each node separately, denoted as Φ(v i ,[VNode]) and Φ([VNode],v i ). The connection between [VNode] and each node is a virtual connection. In order to distinguish the connection between real nodes, Graphomer will and Set as a separately learnable scalar

[0126] S5. Fault tracing

[0127] Fault tracing can be viewed as a binary classification problem, that is, determining whether different nodes are abnormal at the current moment. By converting the predicted values ​​into anomaly scores, nodes can be labeled as normal or abnormal, and a convergence function can be constructed based on the predicted labels of the nodes to train the model.

[0128] The Sigmoid function is a common S-shaped function in biology, also known as the S-shaped growth curve. In information science, due to its monotonic and inverse monotonic properties, the Sigmoid function is often used as an activation function of a neural network to map variables between 0 and 1. The Sigmoid function can be used to calculate the predicted value at time B. Convert to anomaly score:

[0129]

[0130] in, yes The abnormal score of a and b is the weight and bias respectively. If the error value is greater than the limited threshold Υ, the modal data is considered to be faulty:

[0131]

[0132] in, represents the abnormal label of the i-th modal data at time B, and the label set of all modal data is expressed as

[0133] We determine the predicted failure data in different window batches The real fault source F i The mean square error between them is used as the loss function L:

[0134]

[0135] By training the model, the loss function converges. The model can mark abnormal data points, thereby realizing the detection and tracing of abnormalities in industrial production. Algorithm 2: Local anomaly detection algorithm training

[0136]

[0137] This Graphomer-based industrial control system anomaly detection method, by using concurrent association rules to construct association graphs between various data, more effectively represents the association relationship between data, and establishes an industrial system anomaly detection model based on Graphomer; this model can effectively detect anomalies in industrial systems, explain anomalies, and find the source of anomalies. By developing a federated learning framework, each industrial user uses data resources to train the model. In addition, through this framework, the parameters of the same model trained with different data sources can be referenced, which can well protect the security and privacy of industrial data during the training process, thereby constructing a highly secure and accurate anomaly detection model.

[0138] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A graphomer-based industrial control system anomaly detection method, characterized in that: The following steps are involved: S1. System framework workflow S1.

1. System initialization In the initial stage, the cloud server gives initialization parameters based on the anomaly detection model. Including the anomaly detection model parameters q 0 , including the learning rate η, the decay exponent ρ1,ρ2∈[0,1), a small constant ζ for stability, and the loss function Batch size B, contribution rate α k , the number of communication rounds R, the total size of local data N k ; S1.

2. Ontology model training Each industrial agent A k Receive initialization parameters from the cloud server and then use local private data D k Train anomaly detection models; The detailed training process is given in Algorithm 2. Since the industrial agent belongs to the edge server, it has sufficient computing power to support local model training. The training steps of Algorithm 2, i.e., the local anomaly detection algorithm, are as follows: Step 1: Use a sliding window with a specific length and moving distance to convert the local data D k Cut into slice data D X , where x∈X; Step 2: Use association rule mining algorithm to construct the behavior association rule table tab of the data. Step 3: Perform an iterative process to improve the parameters of the model; Step 4: Traverse M different modal data, and for each modal data, calculate the error score between its predicted value and the true value to evaluate the prediction accuracy of the model on different modal data; Step 5: Repeat step 4 until the loss function reaches convergence, that is, the prediction performance of the model no longer improves significantly or reaches the preset number of iterations; the convergence of the loss function means that the model has learned enough information to fit the training data well; Step 6: Return to the latest parameters S1.

3. Model parameter aggregation According to the contribution rate of all industrial agents, the cloud server aggregates all parameters and then sends the aggregated parameters to all industrial agents; S1.

4. Local model update The local agent receives the aggregation parameters of the rth round, uses the same offline local multimodal data to update the model, and re-uploads the updated model parameters; after R rounds of model parameter interaction, the final multimodal anomaly detection model is determined; S2. Construction of data behavior association rules Use sliding windows to slice the collected multimodal data, and build association rule tables between data behaviors based on different combinations of data behaviors under different data slices. S3. Construction of multimodal association graph Select one set of sliced ​​data and construct the association graph between the data according to the association rule table as the input of the Graphomer model; S4. Graphomer-based point prediction In order to detect point anomalies, we use the prediction function of Graphomer to predict the expected behavior at each time point, so as to identify anomalies and find out the source of anomalies. Graphomer is a graph representation learning method that combines Transformer and multiple spatial encoding methods. It adds graph structure information encoding to effectively use the structural information of the graph. There are three types of graph structure information encoding, namely center encoding, spatial encoding and edge encoding. S5. Fault tracing Fault tracing can be viewed as a binary classification problem, that is, determining whether different nodes are abnormal at the current moment. By converting the predicted values ​​into anomaly scores, nodes can be labeled as normal or abnormal, and a convergence function can be constructed based on the predicted labels of the nodes to train the model.

2. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S1, the system framework workflow mainly includes two types of entities, namely, cloud server C and industrial agent A k ,k∈K, connect the industrial agents of multiple industrial control systems to the same cloud server and jointly build an anomaly detection model.

3. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S1, a comprehensive anomaly detection model is established; during initialization, initialization parameters are created. Send it to each industrial agent; in the rth round of training, combine the model parameters trained locally in each industrial agent Then it is sent to various industrial agents.

4. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In the step S1, R rounds (optimal number of rounds obtained from experience) of interaction are performed between the cloud server and each industrial agent, and the final anomaly detection model is obtained through parameter training, aggregation, and interaction.

5. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S2, industrial agent A k The local data in is defined as Use a sliding window with a length of B and a step size of l to intercept data D k , obtain X data slices of length B, where Among them, floor(·) represents the rounding function, D X Represents the total data set after sliding window processing.

6. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S3, any slice data D is selected from the data slices. x ,D x ∈D X As a training batch, in the current data batch In , we take the first B-1 length as the input of the anomaly detection model: x (B) =[s 1 ,s 2 ,…,s (B-1) ]。 7. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S4, in Graphormer, the central encoding is used to represent the importance of the node in the graph, and in the input layer, the node features are and the learnable degree vector Addition; 8. The method for detecting anomalies in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S4, in Graphormer, spatial coding is used to represent the global position relationship of nodes, not only considering the local relationship between the current node and the adjacent nodes, but also calculating the minimum path between the nodes. To represent the node v i ,v j If the two nodes are not related, A i,j Expressed as the i-th row and j-th column element in the Query-Key product matrix A, we have: in is a through The weight matrix can be learned and shared among all layers; h i is the output of the center encoding.

9. The method for detecting abnormality in an industrial control system based on Graphomer according to claim 1, characterized in that: In step S4, in order to better encode the edge features into the attention layer, edge encoding is used to represent the relevance of nodes in Graphormer. First, find two nodes v i ,v j The shortest path between SP i,j =(e1,e2,…,e M ) and calculate the average of the edge feature dot products and the learnable embedding path.