Industrial time series anomaly detection method, device and computer equipment

Abstract

The application provides an industrial time series anomaly detection method, device and computer equipment, and belongs to the technical field of industrial time series anomaly detection. The method comprises the following steps: acquiring industrial time series data; converting the time series data into graph structure data as model input data to input a reconstruction model, to obtain reconstruction data, wherein the reconstruction model comprises a number knowledge composite graph attention network; if the difference degree between the reconstruction variable data of a certain timestamp in the reconstruction data and the original variable data corresponding to the timestamp in the model input data reaches a preset anomaly threshold, it is determined that the original variable data corresponding to the timestamp is in an abnormal state. According to the knowledge graph related to the industrial system, the reconstruction model extracts a superior representation of the variable interaction network based on knowledge, and fuses it with the sequence-based time series dependence, thereby reducing the risk of learning pseudo-correlation by the model, and improving the anomaly detection performance and anomaly explainability of the industrial time series.

Description

Technical Field

[0001] This invention belongs to the field of industrial equipment time series data anomaly detection technology, specifically relating to an industrial time series anomaly detection method, an industrial time series anomaly detection device, a computer device, and a machine-readable storage medium. Background Technology

[0002] Process industries refer to industries that provide raw materials and energy for manufacturing by continuously and complexly altering and reacting resources (including recyclable resources) through physical changes and chemical reactions. These industries typically include chemicals, papermaking, cement, non-ferrous metals, steel, pharmaceuticals, and food and beverage, and are an important part of modern industry. For most process industries, because the input raw materials come from nature or industrial waste, it is difficult to maintain uniformity and consistency in the physical properties of the raw materials, such as pressure, flow rate, and hardness. Therefore, abnormal operating conditions are prone to occur during production, such as abnormal vibration and temperature. If operators fail to detect these abnormalities in a timely manner, it can easily lead to system or equipment malfunctions, shutdowns, or even serious accidents.

[0003] With the development of technologies such as the Internet of Things, big data, and artificial intelligence, the types and quantities of data generated by enterprise production and operation activities are growing exponentially. In recent years, more and more enterprises have begun to collect and analyze this data in real time for visual monitoring of production lines, automated analysis and predictive control of manufacturing processes, to ensure production safety and product delivery efficiency. Anomaly detection in industrial time series data has always been a focus of data analytics applications, especially in process industries. Anomaly warnings and interpretations are crucial for ensuring data quality and maintaining information security and operational stability in process industry manufacturing.

[0004] In the past few years, experienced engineers typically set thresholds manually for each monitored time series to identify anomalies. However, with the increasing volume and number of variables in industrial monitoring systems in recent years, this manual approach is no longer suitable, and data-driven methods have gradually become a research hotspot. Currently, the main problems in industrial time series anomaly detection in the process industry are as follows: First, most purely data-driven methods are susceptible to the quality of training data and require prior knowledge to reduce spurious correlations. In the process industry, due to uncertainties such as raw material type, load, and environmental changes, training data may contain inherent data biases. This leads to data-driven models learning features that have spurious correlations with actual labels. The existence of these features may stem from biases in the collection or generation of training data; they are not themselves decisive factors in identifying anomalies. However, when the test set lacks these feature data, it significantly reduces model performance. Secondly, providing reasonable explanations for detected anomalies is also a challenge. Research on machine learning explanations has become an important aspect of current artificial intelligence. In industrial data anomaly detection, providing reasonable explanations helps determine the causes of anomalies and propose corresponding handling strategies. However, purely data-driven methods often lack the assistance of expert knowledge; their explanations stem from the sensitivity of each module within the entire architecture to the data. Engineers find it difficult to trust explanations without the constraints of natural mechanisms. Thirdly, there is a lack of labeled samples and the diversity of anomalies. Generally, in actual industrial monitoring environments, the number of anomalies is relatively small, while the range of anomaly types is wide, making it difficult to collect enough accurate anomaly label samples for training. Summary of the Invention

[0005] The purpose of this invention is to provide an industrial time series anomaly detection method, an industrial time series anomaly detection device, a computer device, and a machine-readable storage medium to overcome the shortcomings of existing traditional industrial time series anomaly detection methods, such as being greatly affected by spurious correlations and having poor interpretability.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for detecting anomalies in industrial time series data, the method comprising:

[0007] Time series data is obtained by using sensors deployed in the industrial system under test to collect production process data consisting of multiple variables generated during the operation of the industrial system under test.

[0008] The time series data is transformed into a time series graph structure, and the time series graph structure is used as the input data to the pre-built reconstruction model to obtain the reconstruction data.

[0009] If the difference between the reconstructed variable data at a certain timestamp in the reconstructed data and the original variable data corresponding to that timestamp in the model input data reaches a preset abnormal threshold, then the original variable data corresponding to that timestamp is determined to be abnormal.

[0010] The reconstructed model is built with an autoencoder as the baseline, and a first graph attention network layer for extracting high-level knowledge representation information and a second graph attention network layer for extracting temporal high-level information of the model input data are added before the encoding network in the autoencoder. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network in the autoencoder. The input data of the first graph attention network layer is constructed by utilizing the interaction relationship between variables obtained from the knowledge graph between devices and sensors in the industrial system.

[0011] Optionally, the reconstruction model includes a composite encoder, a hidden space for randomly sampling the encoding result of the composite encoder, and a decoder corresponding to the composite encoder constructed based on the variational autoencoder principle. The composite encoder includes a first graph attention network layer, a second graph attention network layer, and an encoding network. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network.

[0012] Optionally, the encoding network is an LSTM network.

[0013] Optionally, the decoding network within the decoder employs an LSTM network.

[0014] Optionally, the degree of difference between the reconstructed variable data at a certain timestamp in the reconstructed data and the original variable data corresponding to that timestamp in the model input data is characterized by an anomaly score, which is calculated using the following formula:

[0015] ;

[0016] in, Represents timestamp i The corresponding abnormal scores, Represents timestamps in the model input data i The corresponding original variable data, Represents timestamps in the reconstructed data i The corresponding reconstructed variable data.

[0017] Optionally, the input data for the first graph attention network layer is constructed using the interaction relationships between variables obtained from the knowledge graph between devices and sensors in the industrial system, including:

[0018] The high-order relationships of the knowledge graph between equipment and sensors in industrial systems are summarized to obtain the direct and potential interaction relationships between various variables;

[0019] Generate a variable interaction network and the adjacency matrix of each node in the variable interaction network based on the direct and potential interaction relationships between variables.

[0020] In the variable interaction network, the attribute value of a node is the observed value of the variable corresponding to that node, and the number of attribute values ​​is determined according to the length of the acquisition time window.

[0021] Optionally, higher-order relationships in a knowledge graph may include at least one of the following: direct relationships between variables and other variables due to their participation in the same technological process or the correlation of data; possible relationships between variables and other variables due to their participation in the same physical process; possible indirect relationships between variables and other variables because their corresponding sensors are installed on the same equipment entity; and possible relationships between variables and other variables because the physical processes involved are interconnected.

[0022] Optionally, the first graph attention network layer employs a multi-head random attention mechanism.

[0023] Optionally, the multi-head random attention mechanism specifically includes:

[0024] Pay attention to the head node separately i The output vector is represented by the following formula: ;

[0025] ;

[0026] ;

[0027] ;

[0028] Nodes are generated by aggregating the output vectors of all individual attention heads. i The final expression And nodes i The final expression This can be expressed by the following formula:

[0029] ;

[0030] in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Random attention scores between () denotes a sampling function based on the Bernoulli distribution. Represents a node i Neighborhood set, Represents a node j The input vector, and Both represent weight matrices. This represents the activation function. Indicates neighboring nodes before normalization. j For nodes i The importance of express The normalization result, This represents the activation function. This represents the feedforward neural function used for inter-node association. Indicates the number of independent attention heads. Indicates the first h A single attention head node i and nodes k Random attention scores between Indicates the first h A single attention head node k The weight matrix corresponding to the input vector.

[0031] Optionally, the nodes of the time series graph structure are timestamps with attribute values ​​within a sliding time window. The observed value of the variable on the timestamp is regarded as the attribute value of the corresponding node. The second graph attention network layer uses a dynamic attention mechanism with a bias mechanism to calculate the feature attention score between timestamps.

[0032] Optional, dynamic attention mechanisms with bias include:

[0033] The output vector of the head node is represented by the following formula:

[0034] ;

[0035] ;

[0036] ;

[0037] in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Dynamic attention scores between them , and Both represent weight matrices. Represents the deviation vector. Represents a node i The adjacency set, Represents a node j The input vector, This represents the activation function. Used to compute nodes before normalization. i to neighboring nodes j Importance score This represents the activation function. This represents the feedforward neural function used for inter-node association.

[0038] Optionally, the step of converting time series data into a time series graph structure includes:

[0039] The time series data is transformed into a sequence data vector by using a sliding time window. The dimension of the data vector is the number of timestamps within the sliding time window, and the length of the data vector is the number of variables.

[0040] The sequence data vector is input into the gated recurrent unit layer to capture the hidden state at each timestamp. The adjacency matrix between data vectors within the sliding time window is calculated using two learnable weights from the linear layer. The adjacency matrix between data vectors within the sliding time window is expressed by the following formula:

[0041] ;

[0042] ;

[0043] ;

[0044] in, Indicates a hidden state. and This represents two learnable weights. This represents the dimension of the data vector. This represents the adjacency matrix between data vectors within the sliding time window.

[0045] Optionally, the method further includes:

[0046] For model input data with abnormal original variable data state and model input data with normal original variable data state, feature attention scores for the first graph attention network layer are calculated respectively, and the calculated feature attention scores are compared to locate the anomaly.

[0047] A second aspect of the present invention provides an industrial time series anomaly detection device, the device comprising:

[0048] The time series data acquisition module is used to collect production process data consisting of multiple variables generated during the operation of the industrial system under test by using sensors deployed in the industrial system under test, and obtain time series data.

[0049] The data reconstruction module is used to transform time series data into a time series graph structure, and input the time series graph structure as the model input data into a pre-built reconstruction model to obtain reconstructed data;

[0050] The anomaly detection module is used to determine whether the difference between the reconstructed variable data at a certain time in the reconstructed data and the original variable data corresponding to that time in the model input data has reached a preset anomaly threshold. If so, the original variable data corresponding to that time is determined to be in an abnormal state.

[0051] The reconstructed model is built with an autoencoder as the baseline, and a first graph attention network layer for extracting high-level knowledge representation information and a second graph attention network layer for extracting temporal high-level information of the model input data are added before the encoding network in the autoencoder. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network in the autoencoder. The input data of the first graph attention network layer is constructed by utilizing the interaction relationship between variables obtained from the knowledge graph between devices and sensors in the industrial system.

[0052] A third aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the industrial time series anomaly detection method described in the first aspect of the present invention.

[0053] A fourth aspect of the present invention provides a machine-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the industrial time series anomaly detection method described in the first aspect of the present invention.

[0054] In the above technical solution, based on the existing knowledge graph between industrial system equipment and sensors, the high-level knowledge representation information of the knowledge-based variable interaction network is extracted using the first graph attention network layer. The extracted high-level knowledge representation information is then fused with the time-series-based high-level information (i.e., time-series dependency information) extracted using the second graph attention network layer. This reduces the risk of the reconstruction model learning spurious correlations, thereby improving the anomaly detection performance for industrial time series and enhancing the interpretability of industrial time series anomalies.

[0055] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0056] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0057] Figure 1 A flowchart illustrating an industrial time series anomaly detection method according to an embodiment of the present invention is shown schematically.

[0058] Figure 2 This diagram illustrates the network architecture of the reconstructed model in a specific application example.

[0059] Figure 3 The illustration shows a comparison of the anomaly detection performance of the CKDGAT, LSTM-AE, LSTM-VAE, DAGMM, OmniAnomaly, USAD, and MTAD-GAT methods on the same time series data in specific application examples.

[0060] Figure 4 This illustration shows a schematic diagram of an abnormal fragment detected in a specific application example;

[0061] Figure 5 The illustration shows the attention score differences for each feature of anomaly segments in a specific application example. Detailed Implementation

[0062] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0063] Example 1

[0064] See Figure 1 The industrial time series anomaly detection method provided by this invention includes the following implementation steps:

[0065] Step S100: Use sensors deployed in the industrial system under test to collect production process data consisting of multiple variables generated during the operation of the industrial system under test, and obtain time series data.

[0066] As we know, time series data refers to a sequence of variable observations at multiple timestamps. Typically, because the variable data collected by sensors in industrial systems is multimodal, for numerical data, the observations can be used directly. For non-numerical data, the variable data can be replaced with equivalent integer values, that is, the equivalent integer values ​​can be used as the corresponding observations of the variable data.

[0067] Step S200: The time series data is transformed into a time series graph structure. This time series graph structure is then input into the pre-built reconstruction model as model input data to obtain the reconstruction data. The reconstruction model is built using an autoencoder as a baseline. A first graph attention network layer (knowledge high-level information extraction graph attention layer) for extracting high-level knowledge representation information and a second graph attention network layer (time-series high-level information extraction graph attention layer) for extracting temporal high-level information from the model input data are added before the encoding network within the autoencoder. The outputs of the first and second graph attention networks are fused and connected to the input of the encoding network within the autoencoder. The input data of the first graph attention network layer is constructed using the interaction relationships between variables obtained from the knowledge graph between devices and sensors in the industrial system.

[0068] For example, the time series can be transformed into a time series graph structure using the transformation method in the general embodiments, and then used as input to the second graph attention network layer.

[0069] In one specific embodiment, converting a time series into a time series graph structure includes:

[0070] The time series data is transformed into a sequence data vector by using a sliding time window. The dimension of the data vector is the number of timestamps within the sliding time window, the length of the data vector is the number of variables, and the position of the variable represents the position of its own observation in the corresponding time dimension.

[0071] The sequence data vectors are input into the Gated Recurrent Unit (GRU) layer, which captures the hidden state at each timestamp sequentially. The adjacency matrix between data vectors within the sliding time window is calculated using two learnable weights from the linear layer. The adjacency matrix between data vectors within the sliding time window is expressed by the following formula:

[0072]

[0073]

[0074] (Formula 1);

[0075] in, Indicates a hidden state. and This represents two learnable weights. The dimension of the data vector. This represents the adjacency matrix between data vectors within the sliding time window.

[0076] Based on the above, it can be seen that the nodes of the time series graph structure are timestamps with attribute values ​​within a sliding time window, and the observed values ​​of variables on the timestamps are regarded as the attribute values ​​of the corresponding nodes.

[0077] For example, the autoencoder may be a variational autoencoder.

[0078] In one specific embodiment, the autoencoder is a variational autoencoder. Accordingly, the reconstruction model includes a composite encoder, a hidden space for randomly sampling the encoding result of the composite encoder, and a decoder corresponding to the composite encoder constructed based on the variational autoencoder principle. The composite encoder includes a first graph attention network layer, a second graph attention network layer, and an encoding network. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network.

[0079] For a reconstruction model using a variational autoencoder as the baseline, see [link to relevant documentation]. Figure 2 As shown. Figure 2 In this model, both the encoding and decoding networks within the decoder employ an LSTM network. The hidden space randomly samples the encoding results from the composite encoder using a Gaussian distribution with a mean of μ and a standard deviation of σ. The composite encoder CE consists of a first graph attention network layer, a second graph attention network layer, and an LSTM network.

[0080] Suppose the input vector of the reconstruction model is represented as x It should follow the conditional distribution. Reconstruction, including z The latent representation is extracted by the composite encoder CE. The goal of reconstructing the model optimization is to find the optimal model parameters so that the reconstructed data matches the latent representation. z The true posterior distributions are very close, that is:

[0081] ;

[0082] In the above formula, Represents the input vector x The reconstruction probability can be expressed by the following formula:

[0083] ;

[0084] Since directly calculating the above equations is difficult, an encoder is used instead. right

[0085] Perform an approximate estimation to obtain the decoder. Its reconstruction loss function can be expressed as:

[0086] ;

[0087] In the above formula, Represents the input vector x The negative log-likelihood expectation; Represents a regularization operator used for computation. and Kullback-Leibler divergence between them For log-likelihoods that are difficult to calculate A practical estimator of the lower bound.

[0088] In one specific embodiment, the input data for the first graph attention network layer is constructed using the interaction relationships between variables obtained from the knowledge graph between devices and sensors in the industrial system, including:

[0089] The high-order relationships of the knowledge graph between equipment and sensors in industrial systems are summarized to obtain the direct and potential interaction relationships between various variables;

[0090] Generate a variable interaction network and the adjacency matrix of each node in the variable interaction network based on the direct and potential interaction relationships between variables.

[0091] In the variable interaction network, the attribute value of a node is the observed value of the variable corresponding to that node, and the number of attribute values ​​is determined according to the length of the acquisition time window.

[0092] For example, the high-order relations of the knowledge graph between devices and sensors can be at least one of the following:

[0093] Direct correlations between variables and other variables due to their participation in the same technological process or the correlation of their data; possible correlations between variables and other variables due to their participation in the same physical process; possible indirect correlations between variables and other variables because their corresponding sensors are installed on the same equipment entity; possible correlations between variables and other variables because the physical processes involved are interconnected.

[0094] It is important to understand that the adjacency matrix can be obtained by calculating the edges of the variable interaction network. The calculation process can be represented as follows:

[0095] (Formula 2);

[0096] in, Represents the adjacency matrix. Represents the adjacency matrix of the nth element. i Line number j The value of the column element, Represents the edge set of a node. Represents a node i With nodes j Does an edge exist between them? ∈ ,but =1, otherwise =0.

[0097] In one specific embodiment, the first graph attention network layer employs a multi-head random attention mechanism. The graph structure information of the first graph attention network layer comes from the adjacency matrix. Since uncertain potential correlations between variables may produce spurious correlations, the introduction of a multi-head random attention mechanism minimizes the impact of spurious correlations, thereby improving the generalization ability of the reconstructed model.

[0098] For example, in the multi-head random attention mechanism used in the first graph attention network layer:

[0099] 1) The output vector of the head node is represented by the following formula:

[0100]

[0101]

[0102]

[0103] (Formula 3);

[0104] in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Random attention scores between () denotes a sampling function based on the Bernoulli distribution. Represents a node i Neighborhood set, Represents a node j The input vector, and Both represent weight matrices. This represents the activation function. Indicates neighboring nodes before normalization. j For nodes i The importance of express The normalization result, This represents the activation function. This represents the feedforward neural function used for inter-node associations. For example, It is the sigmoid activation function.

[0105] 2) Nodes are generated by aggregating the output vectors of all individual attention heads.i The final expression And nodes i The final expression This can be expressed by the following formula:

[0106] (Formula 4);

[0107] in, This indicates that attention should be paid to the number of heads individually. Indicates the first h A single attention head node i and nodes k Random attention scores between Indicates the first h A single attention head node k The weight matrix corresponding to the input vector.

[0108] In one specific embodiment, the second graph attention network layer employs a dynamic attention mechanism with a bias to calculate the feature attention score between timestamps. This dynamic attention mechanism with a bias extracts more accurate high-level temporal information.

[0109] For example, in the dynamic attention mechanism with bias used in the second graph attention network layer: the output vector of the individual attention head node is represented by the following formula:

[0110]

[0111]

[0112] (Formula 5);

[0113] in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Dynamic attention scores between them , and Both represent weight matrices. Represents the deviation vector. Represents a node i The adjacency set, Represents a node j The input vector, This represents the activation function. Used to compute nodes before normalization. i to neighboring nodes jImportance score This represents the activation function. This represents the feedforward neural function used for inter-node associations. For example, It is the sigmoid activation function.

[0114] Step S300: If the difference between the reconstructed variable data at a certain time in the reconstructed data and the original variable data corresponding to that time in the model input data reaches a preset abnormal threshold, then the original variable data corresponding to that time is determined to be abnormal.

[0115] It is known that the abnormal threshold can be determined using algorithms based on percentiles and automatic peak over threshold (POT) algorithms, etc.

[0116] For example, the degree of difference between the reconstructed variable data at a certain timestamp in the reconstructed data and the original variable data corresponding to that timestamp in the model input data is characterized by anomaly score, which is calculated by the following formula:

[0117] (Formula 6);

[0118] in, Represents timestamp i The corresponding abnormal scores, Represents timestamps in the model input data i The corresponding original variable data, Represents timestamps in the reconstructed data i The corresponding reconstructed variable data.

[0119] Example 2

[0120] The difference between this embodiment and Embodiment 1 is that, after step S300, the following steps are also included:

[0121] Step S400 involves calculating feature attention scores for both model input data with abnormal original variable data states and model input data with normal original variable data states, targeting the first graph attention network layer. The calculated feature attention scores are then compared to locate the anomalies. It's important to understand that calculating feature attention scores targeting the first graph attention network layer means applying the multi-head random attention mechanism of the first graph attention network layer to calculate feature attention scores for either the model input data with abnormal original variable data states or the model input data with normal original variable data states.

[0122] As can be seen, feature attention scores characterize the strength of the correlation between variables. If the feature attention score of a variable in an abnormal time series differs significantly from that in a normal time series, it can be inferred that the variable is abnormal. If the feature attention scores of multiple variables differ significantly, specific abnormal phenomena can be inferred and located based on these abnormal variables, thereby realizing the interpretation and location of anomalies. Operation and maintenance personnel can implement corresponding response strategies based on these anomaly interpretation and location results, thereby eliminating anomalies in a timely manner and ensuring the safe and stable operation of industrial systems.

[0123] Example 3

[0124] This embodiment is a specific application example of Embodiment 2. Taking the anomaly detection of monitoring data of the vertical roller mill (VRM), a core equipment in slag grinding production, as an example, it verifies the effectiveness of the above embodiment in anomaly detection. The monitoring data of the vertical roller mill mainly consists of monitoring data of process variables during its operation.

[0125] To overcome the numerous problems encountered by traditional data-driven methods in industrial time series anomaly detection, such as spurious correlations, poor interpretability, and difficulties in integrating knowledge and data from complex systems, Examples 1 and 2 propose a reconstruction model. This model constructs a reconstruction model based on a knowledge-knowledge composite attention network architecture. "Knowledge-knowledge composite" refers to combining the knowledge graph between devices and sensors with the time series data. Specifically, a first-graph attention network layer extracts high-level knowledge representation information, a second-graph attention network layer extracts high-level temporal information from the model's input data, and then the two types of information are fused. An encoding network encodes the fused features. In this application example, the reconstruction model employs the following... Figure 2 The architecture shown, and the method for anomaly detection in the monitoring data of the vertical mill using this reconstruction model, mainly includes the following implementation steps:

[0126] Step A1 involves dividing the vertical mill monitoring data into multiple fixed-length time periods using a sliding time window. The monitoring data for each sliding time window is then converted into a sequence data vector. The timestamp within the time window determines the dimension of the data vector, the number of process variables determines its length, and the position of the process variable indicates the location of its observed value within the corresponding time dimension. Since the monitoring data collected by the sensors from the vertical mill is multimodal, for numerical monitoring data, the observed value of the process variable is directly used. For non-numerical monitoring data, the process variable records are replaced with equivalent integer values ​​to obtain the corresponding observed values.

[0127] Step A2: Input the constructed sequence vector data into the gated recurrent unit layer to capture the hidden state at each timestamp in sequence. SThe hidden state contains information about the data that the network has processed. This is achieved through Equation 1, utilizing the hidden state. S and two learnable weights and The adjacency matrix of the time series within the sliding time window is calculated. Steps A1 and A2 transform the time series within the sliding time window into a time series graph structure. The nodes in the time series graph structure are timestamps with attribute values, and the observed values ​​of process variables at the timestamps are regarded as the attribute values ​​of the corresponding nodes.

[0128] Step A3 involves summarizing the high-order relationships of the knowledge graph between the various equipment components on the vertical mill and the sensors installed on them to obtain the direct and potential interaction relationships between process variables. This constructs a variable interaction network, and the adjacency matrix between nodes in this network is determined. The attribute values ​​of the nodes originate from the observed values ​​of process variables collected by the sensors, and the number of node attribute values ​​is determined by the length of the acquisition time window. Using Equation 2, the adjacency matrix is ​​obtained by calculating the edges in the variable interaction network. Step A3 transforms the complex interactions between various process variables within the vertical mill into a graphical structure.

[0129] In this step, the higher-order relationships of the knowledge graph include: 1) a variable is directly related to other variables because they participate in the same process or their data are related; 2) a variable may be related to other variables because they all participate in a physical process; 3) a variable may be indirectly related to other variables because their sensors are mounted on a device component entity; 4) a variable may be related to another variable because the physical processes they involve are interconnected.

[0130] Step A4, use the adjacency matrix obtained in step A3. This serves as the input data for the first graph attention network layer. The first graph attention network layer uses a multi-head random attention mechanism to calculate the feature attention scores between variables. The principle of the multi-head random attention mechanism is shown in Equations 3 and 4.

[0131] Step A5: Transform the time series within the sliding time window into the adjacency matrix. This serves as the input data for the second graph attention network model layer. The second graph attention network layer uses a dynamic attention mechanism with bias to calculate the feature attention score between timestamps. The principle of the dynamic attention mechanism with bias is shown in Equation 5.

[0132] Step A6: The outputs of the first graph attention network layer and the second graph attention network layer are fused and then connected to the input of the LSTM network. For ease of distinction, the LSTM network connected to the fused output of the first and second graph attention network layers is defined as the first LSTM network. The latent representation is then encoded using the first LSTM network. z .

[0133] Step A7: The hidden space is randomly sampled from the actual encoding of the first LSTM network output using a Gaussian distribution. The mean of the Gaussian distribution is μ, and the standard deviation is σ. Figure 2 In this context, Sampling means sampling.

[0134] Step A8: The decoder includes an LSTM network and a linear layer connected in sequence. For ease of distinction, the LSTM network in the decoder is defined as the second LSTM network. The decoder decodes the sampled code to obtain the reconstructed data.

[0135] Step A9: Calculate the anomaly score using Equation 6, and use a preset anomaly threshold for anomaly judgment. The judgment rule is: if the anomaly score corresponding to a certain timestamp reaches the anomaly threshold, then the original variable data of that timestamp is classified as "abnormal"; otherwise, the original variable data of that timestamp is classified as "normal".

[0136] Step A10: For model input data with abnormal original variable data state and model input data with normal original variable data state, calculate the feature attention score for the first graph attention network layer respectively, and compare the calculated feature attention scores to locate the anomaly.

[0137] In this application example, the industrial time series anomaly detection methods implemented in Examples 1 and 2 are defined as the CKDGAT method. The anomaly detection performance of this method compared with other methods is shown in Table 1 and... Figure 3 As shown in Table 1, other methods listed include: LSTM-AE, LSTM-VAE, DAGMM, OmniAnomaly, USAD, and MTAD-GAT. Table 1 compares the evaluation metrics obtained from two different anomaly thresholding algorithms, including precision, recall, F1-score, and Auc. In Table 1, Prec, Rec, F1, and Auc correspond to the evaluation metrics of the percentile thresholding algorithms. , , , The corresponding evaluation metrics are for the automatic peak exceedance (POT) algorithm. As shown in Table 1, the anomaly detection performance based on the CKDGAT method is significantly better than other methods.

[0138] Table 1

[0139]

[0140] like Figures 4 to 5 As shown, a time series segment of anomaly detection was selected for anomaly interpretation and localization. The anomaly segment is as follows: Figure 4 The yellow area marked in the middle. Figure 5 In response to Figure 4 A schematic diagram illustrating the feature attention score difference calculated from anomaly fragments, by... Figure 5 It can be seen that the attention score differences of features L0014, L0050, L0034, L0016, and M0001 rank in the top five. Among them, L0014 and L0016 represent the main motor winding temperature, L0050 represents the vibration measures of the VRM, L0034 represents the hot air valve opening feedback used for manual control, and M0001 represents the total material feed rate. Based on process knowledge, we can infer that the increase in motor load is likely the cause of the subsequent temperature rise, leading to a change in the vibration mode and ultimately causing the anomaly. Therefore, maintenance engineers can achieve the control objective by adjusting the opening of the hot air valve.

[0141] Example 4

[0142] This invention provides an industrial time series anomaly detection device, comprising a time series data acquisition module, a data reconstruction module, and an anomaly judgment module, wherein:

[0143] The time series data acquisition module is used to collect production process data consisting of multiple variables generated during the operation of the industrial system under test by using sensors deployed in the industrial system under test, and obtain time series data.

[0144] The data reconstruction module is used to transform time series data into a time series graph structure, and input the time series graph structure as the model input data into a pre-built reconstruction model to obtain reconstructed data;

[0145] The anomaly detection module is used to determine whether the difference between the reconstructed variable data at a certain time in the reconstructed data and the original variable data corresponding to that time in the model input data has reached a preset anomaly threshold. If so, the original variable data corresponding to that time is determined to be in an abnormal state.

[0146] The reconstructed model is built with an autoencoder as the baseline, and a first graph attention network layer for extracting high-level knowledge representation information and a second graph attention network layer for extracting temporal high-level information of the model input data are added before the encoding network in the autoencoder. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network in the autoencoder. The input data of the first graph attention network layer is constructed by utilizing the interaction relationship between variables obtained from the knowledge graph between devices and sensors in the industrial system.

[0147] In one specific embodiment, the reconstruction model includes a composite encoder, a hidden space for randomly sampling the encoding result of the composite encoder, and a decoder corresponding to the composite encoder constructed based on the variational autoencoder principle. The composite encoder includes a first graph attention network layer, a second graph attention network layer, and an encoding network. The outputs of the first graph attention network layer and the second graph attention network layer are fused and then connected to the input of the encoding network.

[0148] In one specific embodiment, the encoding network employs an LSTM network.

[0149] In one specific embodiment, the decoding network within the decoder employs an LSTM network.

[0150] In one specific embodiment, the degree of difference between the reconstructed variable data at a certain timestamp in the reconstructed data and the original variable data corresponding to that timestamp in the model input data is characterized by an anomaly score, which is calculated by the following formula:

[0151] ;

[0152] in, Represents timestamp i The corresponding abnormal scores, Represents timestamps in the model input data i The corresponding original variable data, Represents timestamps in the reconstructed data i The corresponding reconstructed variable data.

[0153] In one specific embodiment, the input data for the first graph attention network layer is constructed using the interaction relationships between variables obtained from the knowledge graph between devices and sensors in the industrial system, including:

[0154] The high-order relationships of the knowledge graph between equipment and sensors in industrial systems are summarized to obtain the direct and potential interaction relationships between various variables;

[0155] Generate a variable interaction network and the adjacency matrix of each node in the variable interaction network based on the direct and potential interaction relationships between variables.

[0156] In the variable interaction network, the attribute value of a node is the observed value of the variable corresponding to that node, and the number of attribute values ​​is determined according to the length of the acquisition time window.

[0157] In one specific embodiment, the higher-order relationships of the knowledge graph include at least one of the following: direct correlation between variables and other variables due to their participation in the same process or the correlation of data; possible correlation between variables and other variables due to their participation in the same physical process; possible indirect relationship between variables and other variables because their corresponding sensors are installed on the same equipment entity; and possible correlation between variables and other variables because the physical processes involved are interconnected.

[0158] In one specific embodiment, the first graph attention network layer employs a multi-head random attention mechanism.

[0159] In one specific embodiment, the multi-head random attention mechanism is as follows:

[0160] Pay attention to the head node separately i The output vector is represented by the following formula: ;

[0161] ;

[0162] ;

[0163] ;

[0164] Nodes are generated by aggregating the output vectors of all individual attention heads. i The final expression And nodes i The final expression This can be expressed by the following formula:

[0165] ;

[0166] in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Random attention scores between () denotes a sampling function based on the Bernoulli distribution. Represents a node i Neighborhood set, Represents a node j The input vector, and Both represent weight matrices. This represents the activation function. Indicates neighboring nodes before normalization. j For nodes i The importance of express The normalization result, This represents the activation function. This represents the feedforward neural function used for inter-node association. Indicates the number of independent attention heads. Indicates the first h A single attention head node i and nodes k Random attention scores between Indicates the first h A single attention head node k The weight matrix corresponding to the input vector.

[0167] In one specific embodiment, the nodes of the time series graph structure are timestamps with attribute values ​​within a sliding time window. The observed values ​​of variables on the timestamps are regarded as the attribute values ​​of the corresponding nodes. The second graph attention network layer uses a dynamic attention mechanism with a bias mechanism to calculate the feature attention scores between timestamps.

[0168] In one specific embodiment, the dynamic attention mechanism with bias is as follows:

[0169] The output vector of the head node is represented by the following formula:

[0170] ;

[0171] ;

[0172] ;

[0173] in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Dynamic attention scores between them , and Both represent weight matrices. Represents the deviation vector. Represents a nodei The adjacency set, Represents a node j The input vector, This represents the activation function. Used to compute nodes before normalization. i to neighboring nodes j Importance score This represents the activation function. This represents the feedforward neural function used for inter-node association.

[0174] In one specific embodiment, the step of converting time series data into a time series graph structure includes:

[0175] The time series data is transformed into a sequence data vector by using a sliding time window. The dimension of the data vector is the number of timestamps within the sliding time window, and the length of the data vector is the number of variables.

[0176] The sequence data vector is input into the gated recurrent unit layer to capture the hidden state at each timestamp. The adjacency matrix between data vectors within the sliding time window is calculated using two learnable weights from the linear layer. The adjacency matrix between data vectors within the sliding time window is expressed by the following formula:

[0177] ;

[0178] ;

[0179] ;

[0180] in, Indicates a hidden state. and This represents two learnable weights. This represents the dimension of the data vector. This represents the adjacency matrix between data vectors within the sliding time window.

[0181] The device further includes an anomaly interpretation and localization module, which is used to calculate feature attention scores for the first graph attention network layer for model input data with anomalies in the original variable data state and model input data with normal original variable data state, and to locate the anomaly by comparing the calculated feature attention scores.

[0182] On the other hand, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described industrial time series anomaly detection method.

[0183] In another aspect, embodiments of the present invention also provide a machine-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described industrial time series anomaly detection method.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0186] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial time series anomaly detection method, characterized by, The method comprises: Collecting production process data composed of multiple variables generated in the operation of the industrial system to be detected by using sensors arranged in the industrial system to be detected to obtain time series data; Converting the time series data into a time series graph structure, inputting the time series graph structure as model input data into a pre-constructed reconstruction model to obtain reconstruction data; If the difference between the reconstruction variable data of a certain timestamp in the reconstruction data and the original variable data corresponding to the timestamp in the model input data reaches a preset abnormal threshold, it is determined that the original variable data corresponding to the timestamp is abnormal; The reconstruction model is constructed based on a self-encoder, and a first graph attention network layer for extracting high-level knowledge representation information and a second graph attention network layer for extracting high-level information of the time sequence of the model input data are added in front of the encoding network in the self-encoder. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network in the self-encoder. The input data of the first graph attention network layer is constructed by using the interaction relationship between variables obtained from the knowledge graph between devices and sensors in the industrial system.

2. The industrial time series anomaly detection method of claim 1, wherein, The reconstruction model comprises a composite encoder, a hidden space for randomly sampling the encoding result of the composite encoder, and a decoder corresponding to the composite encoder constructed based on the principle of variational auto-encoding. The composite encoder comprises a first graph attention network layer, a second graph attention network layer, and an encoding network. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network.

3. The industrial time series anomaly detection method according to claim 1 or 2, characterized in that, The encoding network adopts an LSTM network.

4. The industrial time series anomaly detection method of claim 2, wherein, The decoding network in the decoder adopts an LSTM network.

5. The industrial time series anomaly detection method of claim 1, wherein, The difference between the reconstruction variable data of a certain timestamp in the reconstruction data and the original variable data corresponding to the timestamp in the model input data is represented by an anomaly score, which is calculated by the following formula: ； wherein, representing a timestamp i corresponding abnormal score, representing a timestamp in model input data i corresponding raw variable data, representing a timestamp in reconstructed data i corresponding reconstructed variable data.

6. The industrial time series anomaly detection method of claim 1, wherein, The input data of the first graph attention network layer is constructed by using the interaction relationship between variables obtained from the knowledge graph between devices and sensors in the industrial system, which comprises: Summarizing the high-order relationships of the knowledge graph between devices and sensors in the industrial system to obtain direct interaction relationships and potential interaction relationships between variables; Generating a variable interaction network and an adjacency matrix of each node in the variable interaction network according to the direct interaction relationships and the potential interaction relationships between variables; The attribute value of a node in the variable interaction network is the observation value of the variable corresponding to the node, and the number of attribute values is determined according to the length of the collection time window.

7. The industrial time series anomaly detection method of claim 6, wherein, The high-order relationships of the knowledge graph include at least one of the following: a direct correlation relationship between variables and other variables due to participation in the same process or data correlation, a possible correlation relationship between variables and other variables due to participation in the same physical process, a possible indirect relationship between variables and other variables due to the corresponding sensors being installed on the same device entity, and a possible correlation relationship between variables and other variables due to the involvement of the physical processes being related to each other.

8. The industrial time series anomaly detection method of claim 6, wherein, The first graph attention network layer adopts a multi-head random attention mechanism.

9. The industrial time series anomaly detection method of claim 8, wherein, The multi-head random attention mechanism is specifically: Pay attention to the head node separately i The output vector is represented by the following formula: ; ； ； ； By aggregating the output vectors of all individual attention heads, the final representation of the node i is produced and the final representation of the node i is represented by ​ ； in, Represents a node i The input vector, Represents a node i The output vector, Represents a node i and nodes j Random attention scores between () denotes a sampling function based on the Bernoulli distribution. Represents a node i Neighborhood set, Represents a node j The input vector, and Both represent weight matrices. This represents the activation function. Indicates neighboring nodes before normalization. j For nodes i The importance of express The normalization result, This represents the activation function. This represents the feedforward neural function used for inter-node association. Indicates the number of independent attention heads. Indicates the first h A single attention head node i and nodes k Random attention scores between Indicates the first h A single attention head node k The weight matrix corresponding to the input vector.

10. The industrial time series anomaly detection method of claim 1, wherein, The node of the time series graph structure is a timestamp with an attribute value in a sliding time window, and the observation value of a variable on the timestamp is regarded as the attribute value of the corresponding node. The second graph attention network layer adopts a dynamic attention mechanism with a bias mechanism to calculate the feature attention score between the timestamps.

11. The industrial time series anomaly detection method of claim 10, wherein, The dynamic attention mechanism with a bias mechanism is specifically: The output vector of the individual attention head node is represented by the following formula: ； ； ； wherein, represents an input vector of a node i , represents an output vector of a node i , represents a dynamic attention score between a node i and a node j , , and each represent a weight matrix, represents a bias vector, represents a neighborhood set of a node i , represents an input vector of a node j , represents an activation function, is used to calculate an importance score of a node i to a neighbor node j before normalization processing, represents an activation function, represents a feed-forward neural function for inter-node association.

12. The industrial time series anomaly detection method of claim 1, wherein, The time series data is converted into a time series graph structure, including: The time series data is converted into a sequence data vector through a sliding time window. The dimension of the data vector is the number of timestamps in the sliding time window, and the length of the data vector is the number of variables. The sequence data vector is input into the gated recurrent unit layer to sequentially capture the hidden state at each timestamp, and a linear layer is used to calculate the adjacency matrix between the data vectors in the sliding time window through two learnable weights. The adjacency matrix between the data vectors in the sliding time window is represented by the following formula: ； ； ； wherein, denotes a hidden state, and denotes two learnable weights, denotes the dimension of the data vector, denotes an adjacency matrix between data vectors within a sliding time window.

13. The industrial time series anomaly detection method of claim 1, wherein, The method further includes: The feature attention score calculation of the first graph attention network layer is performed on the model input data with an abnormal original variable data state and the model input data with a normal original variable data state, respectively. The calculated feature attention scores are compared to locate the abnormality.

14. An industrial time series anomaly detection apparatus characterized by comprising: The device includes: A time series data acquisition module is configured to acquire production process data composed of multiple variables generated during the operation of the industrial system to be detected by using sensors arranged in the industrial system to be detected, and obtain time series data. A data reconstruction module is configured to convert the time series data into a time series graph structure, and input the time series graph structure as model input data into a pre-constructed reconstruction model to obtain reconstructed data. An abnormality judgment module is configured to judge whether the difference between the reconstructed variable data of a certain timestamp in the reconstructed data and the original variable data corresponding to the timestamp in the model input data reaches a preset abnormality threshold. If yes, it is determined that the original variable data corresponding to the timestamp is abnormal. The reconstruction model is constructed based on a self-encoder, and a first graph attention network layer for extracting high-level knowledge representation information and a second graph attention network layer for extracting high-level information of the model input data are added in front of the encoding network in the self-encoder. The outputs of the first graph attention network layer and the second graph attention network layer are fused and connected to the input of the encoding network in the self-encoder. The input data of the first graph attention network layer is constructed by using the interaction relationship between variables obtained from the knowledge graph between devices and sensors in the industrial system.

15. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the program to implement the industrial time series anomaly detection method of any one of claims 1-13.

16. A machine-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the industrial time series anomaly detection method of any one of claims 1-13.

Images