Cyber-physical system anomaly state detection method and related components

Abstract

This application relates to a model construction method, a multivariate time series anomaly detection method, and related components, belonging to the field of system anomaly detection technology. The method includes: initializing learnable embedding vectors for device nodes; determining hyperedges of a first hypergraph structure based on the similarity between learnable embedding vectors; adjusting the device nodes and hyperedges in the first hypergraph structure based on an approximate discrete optimization mechanism to obtain a second hypergraph structure; modeling the learnable embedding vectors, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model; combining the spatiotemporal hypergraph neural network model and a fully connected neural network to generate an initial anomaly detection model; and training and validating the initial anomaly detection model using a target dataset and a mean squared error loss function to generate a target anomaly detection model. This application can improve detection efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of system anomaly detection technology, and in particular to a method and related components for detecting abnormal states in cyber-physical systems. Background Technology

[0002] With the rapid popularization of cyber-physical systems (CPS) such as industrial systems, data centers, the Internet of Things, and vehicle-to-everything (V2X) networks, the monitoring and protection of CPS systems are becoming increasingly important, especially for critical application infrastructures such as power grid systems, water resource monitoring systems, and transportation systems.

[0003] The above-mentioned systems are generally equipped with many different (heterogeneous) sensors to detect abnormal states of the system. For example, in water resource systems, there are sensors that detect water level, flow rate, water quality and temperature. The data obtained from these monitoring devices or sensors will form complex multivariate time series (MTS). Automatically detecting abnormal states of the system from these complex sequence data is a difficult task. The difficulties are: (1) Different time series may be collected by different types of sensors, and the changes between sensors may be correlated in time and space, making it difficult to analyze the complex nonlinear relationships in a simple way; (2) Some systems may have hundreds or thousands of CPS subsystems, each of which records a number of time series data, making the total amount of data exceed the limit of manual management; (3) It is difficult to collect the system state when a large amount of abnormal information occurs, that is, the abnormality of the system must be detected in an unsupervised manner. Summary of the Invention

[0004] Therefore, it is necessary to provide a model construction method, a multivariate time series anomaly detection method, and related components that can effectively mine complex interaction relationships within multivariate time series to better complete the anomaly detection task, addressing the aforementioned technical problems.

[0005] Firstly, a method for detecting anomalies in multivariate time series data is provided, the method comprising:

[0006] Initialize the learnable embedding vectors of the device nodes, and determine the hyperedges of the first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges;

[0007] Based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure;

[0008] The learnable embedding vector, the second hypergraph structure, and the multivariate time series are modeled to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device node;

[0009] By combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model is generated. The fully connected neural network is used to predict the state of the device node at the next time step based on the output value of the spatiotemporal hypergraph neural network model.

[0010] The initial anomaly detection model is trained and validated using the target dataset and the mean squared error loss function to generate the target anomaly detection model.

[0011] Optionally, the hyperedges of the first hypergraph structure are determined based on the similarity between learnable embedding vectors, including:

[0012] Using a vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated.

[0013] The similarity scores are sorted from largest to smallest to obtain the sorting results, and the target number of device nodes are selected based on the sorting results.

[0014] With the device node corresponding to the target learnable embedding vector as the center, construct a hyperedge containing the number of device nodes of the target;

[0015] The first hypergraph structure is generated based on device nodes and hyperedges.

[0016] Optionally, based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure, including:

[0017] Initialize the learnable sampling parameters of the hyperedge;

[0018] Based on the discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0019] Optionally, based on an approximate discrete optimization mechanism, adjusting the device nodes and hyperedges in the first hypergraph structure to obtain the second hypergraph structure further includes:

[0020] Define the hidden state of the device node and the hidden state of the superedge;

[0021] Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge;

[0022] Based on the similarity, determine the increment matrix and mask matrix of the hyperedge;

[0023] Based on the hyperedge mask, the hyperedge increment matrix, and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain the second hypergraph structure.

[0024] Optionally, modeling the learnable embedding vectors, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model includes:

[0025] Based on the learnable embedding vectors of the device nodes and the multivariate time series, the feature matrix of the device nodes is determined;

[0026] A hypergraph neural network is determined to characterize the hypergraph data, wherein the hypergraph data is a time series composed of the device node feature matrix and the second hypergraph structure;

[0027] Determine an attention mechanism model for processing the time series;

[0028] The spatiotemporal hypergraph neural network model is generated by combining the hypergraph neural network and the attention mechanism model.

[0029] Optionally, methods for representing hypergraph data based on hypergraph neural networks include:

[0030] Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges;

[0031] Aggregate the features of the hyperedge to the corresponding device node;

[0032] Repeatedly iterate the aggregation operation to obtain the hidden state of nodes and hyperedges at the target time within the target time window.

[0033] Optionally, the method further includes:

[0034] The hidden states of nodes and hyperedges at multiple target times within the target time window are aggregated to obtain the time series of the spatial relationships of nodes and hyperedges in the second hypergraph structure within the target time window.

[0035] Optionally, methods for processing time series based on attention mechanism models include:

[0036] Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0037] Optionally, the initial anomaly detection model is trained and validated using the target dataset and the mean squared error loss function to generate the target anomaly detection model, including:

[0038] The target dataset is divided into a training set and a validation set;

[0039] The initial anomaly detection model is trained and validated based on the adaptive optimizer, the training set, and the validation set, and the loss value corresponding to the validation set is determined based on the mean squared error loss function.

[0040] Based on the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and then the target anomaly detection model is output.

[0041] Secondly, a multivariate time series anomaly detection method is provided, the method comprising:

[0042] Obtain multivariate time series data of device nodes in the system under test;

[0043] The multivariate time series data is input into the target anomaly detection model constructed by the above method, and the predicted value is output.

[0044] The deviation between the predicted value and the actual value is calculated, and the state of the system to be detected is determined based on the comparison between the first target deviation value and the first preset threshold.

[0045] Optionally, after calculating the deviation between the predicted value and the actual value, the method further includes:

[0046] The deviation value is robustly normalized to obtain the deviation value of the first target node;

[0047] The deviation values ​​of multiple first target nodes at the same time point of the system are sorted, and the first target deviation value is determined based on the sorting results.

[0048] Optionally, determining the state of the system to be detected based on the comparison between the first target deviation value and the first preset threshold includes:

[0049] Obtain the deviation value of the second target node obtained during the validation of the initial anomaly detection model;

[0050] The deviation values ​​of the second target node corresponding to multiple time nodes within the target time window are sorted, and the first preset threshold is determined based on the sorting results.

[0051] When the first target deviation value is detected to be greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0052] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model building method provided in the first aspect and / or the multivariate time series anomaly detection method provided in the second aspect.

[0053] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the model building method provided in the first aspect and / or the multivariate time series anomaly detection method provided in the second aspect.

[0054] Fifthly, a computer program product is provided, the computer program product comprising a computer program, which, when executed by a processor, implements the model building method provided in the first aspect and / or the multivariate time series anomaly detection method provided in the second method.

[0055] The aforementioned model construction method, multivariate time series anomaly detection method, and related components, by introducing hypergraph structures into the multivariate time series anomaly detection problem, can better characterize the complex high-order interaction relationships between different device nodes in a real system, which is beneficial for fitting the system's evolution. By using spatiotemporal hypergraph neural networks to model multivariate time series, spatial relationships and temporal information are modeled, thereby better completing the multivariate time series anomaly detection task and improving detection efficiency and accuracy. Attached Figure Description

[0056] Figure 1 (a)-1(b) are schematic diagrams of a hypergraph and paired interaction simple graphs in one embodiment;

[0057] Figure 2 This is an application environment diagram of a multivariate time series anomaly detection method in one embodiment;

[0058] Figure 3 This is a flowchart illustrating a model building method in one embodiment;

[0059] Figure 4 This is another flowchart illustrating the model building method in one embodiment;

[0060] Figure 5 This is a flowchart illustrating a multivariate time series anomaly detection method in one embodiment;

[0061] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] It should be understood that, in the description of this application, unless the context explicitly requires it, words such as "including" or "comprising" throughout the specification should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0064] It should also be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0065] It should be noted that the terms "S1," "S2," etc., are used only for descriptive purposes and do not specifically refer to the order or sequence, nor are they intended to limit this application. They are merely for the convenience of describing the method of this application and should not be construed as indicating the sequential order of the steps. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0066] As the background technology indicates, there is currently a problem of difficulty in automatically detecting abnormal states in systems from complex sequence data. To solve this problem, existing technologies use graphs to capture the complex relationships between multivariate time series. Existing solution 1 randomly initializes a learnable embedding vector for each time series, then uses KNN (K-Nearest Neighbor) to construct a graph structure based on the embedding vector. Then, based on this graph structure, a graph attention neural network is used to fuse the temporal information of nodes and the embedding vector to predict the future state of each node, thereby achieving the goal of prediction-based anomaly detection. The subsequent specific process includes using the embedding vector based on KNN to generate graph structures for different nodes (sensors), and then using... Graph structure, time series data, and embedding vectors are used to complete time series prediction, followed by anomaly detection based on the prediction. Existing scheme 2 improves upon scheme 1 by making corresponding improvements at each step, including: 1. Adding learnable parameters during the graph structure generation process using embedding vectors; 2. Using a temporal convolutional network (TGN) to encode the time series of each node; 3. Combining the graph structure and TGN, using a spatiotemporal graph convolutional network (STGNN) to represent the existing time series and output predictions for the time series; 4. Using PCA (principal component analysis) instead of simple absolute distance when calculating the difference between predicted and observed values. However, the above methods have the following problems: 1. They all assume that nodes interact directly in pairs, thus using a graph to model the relationships between nodes. However, this prior assumption is unreasonable. For example, ... Figure 1 (a) and Figure 1 (b) The hypergraph and the simple graph with paired interactions are shown respectively. In the simple graph with paired interactions, each edge can only connect 2 nodes. In the hypergraph, a hyperedge can contain any number of nodes. Specifically, in the water resource monitoring system, Four sensors work together to monitor the condition of a reservoir, and Three sensors jointly monitor a river. The first four sensors are collectively affected by the higher-order factor of the "reservoir," while the latter three sensors are collectively affected by the higher-order factor of the "river." This creates a higher-order correlation between the sensors, which cannot be characterized by pairwise interactions. In the above scheme, this correlation is represented by... Fully connected graphs and Instead of representing them as fully connected graphs, a more reasonable approach is to use hypergraphs, which... Place it in a superedge representing a "reservoir". 1. Placed within a hyperedge representing a "river"; 2. Existing methods are too coarse in constructing graph structures and cannot flexibly adjust the graph structure. They can only rely on KNN to connect each node to its K nearest neighbors to construct the graph. This graph construction method cannot capture and characterize the relationship between multivariate time series to the greatest extent. For example, some nodes are completely independent of other nodes. Connecting them to K neighbor nodes will introduce noise into the prediction of that node; 3. The existing time series of each node is very important for state prediction. However, existing solution 1 directly ignores the time series information, while existing solution 2 only uses TCN (Temporal Convolutional Networks, a convolutional neural network architecture designed for sequence modeling) to capture the time series information. It cannot effectively detect abnormal results based on this time series information.

[0067] To address the aforementioned technical issues, this application provides a model construction method, a multivariate time series anomaly detection method, and related components. A hypergraph is used to represent higher-order dependencies between nodes. By dynamically adjusting the hypergraph structure, noisy nodes are removed from hyperedges, and effective nodes are added to hyperedges, resulting in a more flexible and effective hypergraph structure. This provides a foundation for accurate prediction of anomalies. Furthermore, by constructing a more effective time series model to capture the time series information of each node, the accuracy and efficiency of multivariate time series anomaly detection are improved.

[0068] The model building method and multivariate time series anomaly detection method provided in this application can be applied to, for example... Figure 2 In the application environment shown, terminal 102 communicates with a data processing platform set on server 104 via a network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 104 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0069] In one embodiment, such as Figure 3 As shown, a model building method is provided, which can be applied to... Figure 2 Taking the terminal in the example, the explanation includes the following steps:

[0070] S101. Initialize the learnable embedding vectors of the device nodes, and determine the hyperedges of the first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges.

[0071] It should be noted that a device node can refer to a device in a Cyber-Physical System (CPS) or a sensor on a device; the learnable embedding vector is obtained by randomly initializing the device node; a hyperedge consists of multiple device nodes; the hypergraph structure consists of a set of nodes and a set of hyperedges, and each node can carry additional information, including node embedding information and time series information, which can be collectively referred to as node features.

[0072] In some specific implementations, determining the hyperedges of the first hypergraph structure based on the similarity between learnable embedding vectors includes:

[0073] Using a vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated. In CPS systems, node heterogeneity is significant; different nodes may represent different device types and operate in different environments. To automatically capture this node heterogeneity, this application adaptively learns the features of different nodes using learnable node embeddings and uses these embeddings for subsequent hypergraph structure learning. Specifically, for... Each device node is randomly initialized in this application. The learnable node embedding vectors are ,in , indicating device node of 3D embedding vectors can be optimized using gradient descent. Optimization is performed; furthermore, this application utilizes a learnable hypergraph structure to capture high-order interactions between device nodes. The learning and optimization of the hypergraph structure requires starting with a coarse hypergraph structure, referred to as hypergraph structure initialization. This initialization is achieved by using node embedding vectors from the steps described above to obtain a first hypergraph structure. Specifically, a vector similarity function is used to calculate the interaction between each node pair. The similarity is the similarity between the target learnable embedding vector and other learnable embedding vectors. The vector similarity function can be chosen according to actual needs, such as cosine similarity, which calculates the cosine similarity between the embedding vectors of two nodes. When using cosine similarity to calculate similarity, the expression for cosine similarity is: ,in, Indicates similarity;

[0074] The similarity scores are sorted from highest to lowest to obtain the ranking results. Based on these ranking results, device nodes corresponding to the highest number of target similarities are selected. A higher similarity value indicates higher similarity; therefore, device nodes with the highest similarity scores are selected to construct hyperedges. You can choose according to your actual needs;

[0075] Centered on the device node corresponding to the target learnable embedding vector, construct a hyperedge containing the number of device nodes corresponding to the target. Specifically, when the device node corresponding to the target learnable embedding vector is... At that time, with each node Build a center containing The superedge of each node The hyperedge contains nodes Highest similarity Each node can be further used to obtain the set of all initialized hyperedges. ;

[0076] The first hypergraph structure is generated based on device nodes and hyperedges. Specifically, the hypergraph structure can be initialized and the first hypergraph structure generated based on the set of device nodes and the set of hyperedges. The number of device nodes and the number of hyperedges are both... For ease of representation, an incidence matrix can be used. To represent a hypergraph structure:

[0077]

[0078] The first hypergraph structure is represented as .

[0079] In the above implementation, by introducing the hypergraph structure into the anomaly detection problem of multivariate time series, the complex high-order interaction relationships between different devices in the real system can be better characterized. This is beneficial for fitting the evolution of the CPS system, thereby obtaining more accurate system state prediction and anomaly detection results, and also helps maintenance personnel understand the system's operating mechanism and the root cause of system anomalies.

[0080] S102. Based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure.

[0081] It should be noted that the approximate discrete optimization mechanism includes hyperedge sampling and node sampling, used to optimize and adjust device nodes and hyperedges. Hyperedge optimization is used to remove redundant hyperedges introduced during hypergraph initialization, while node optimization is used to remove redundant nodes within hyperedges and include hidden nodes not included in the hyperedges, further optimizing the first hypergraph structure. Specifically, the first hypergraph structure initialized using TopK (finding the k most important or highest-ranking elements from a set containing a large number of data items) cannot characterize the heterogeneity of each device node in a fine-grained manner. For example, for completely independent nodes (sensors...), ... If a node (or device) is placed inside any hyperedge, the time series prediction of that node will contain noise from other nodes. In addition, for some higher-order interactions, such as weather factors, the state of more than k nodes will be affected. Limiting the size of the hyperedge to k will impair the model's ability to capture such higher-order interactions. Therefore, it is necessary to automatically remove nodes from unsuitable hyperedges through higher-order structure learning and put the nodes missed by the TopK method in step S102 into the corresponding hyperedges. Based on this, the above objectives are achieved through an approximate discrete optimization mechanism composed of hyperedge sampling and node sampling.

[0082] In some specific implementations, based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure, including:

[0083] Initialize the learnable sampling parameters for the hyperedges; specifically, initialize one learnable sampling parameter for each hyperedge in the first hypergraph structure. , represented as This parameter can be optimized through gradient optimization;

[0084] Based on a discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask. Here, the discrete distribution sampling mechanism refers to the Gumbel sampling mechanism; that is, to ensure differentiability, Gumbel sampling is used to complete the process from... get The process For each superedge The hyperedge mask is expressed as follows:

[0085]

[0086] in, , This refers to the sigmod function. Indicates temperature parameter;

[0087] The mask for sampling the hyperedges of the first hypergraph structure is represented as:

[0088]

[0089] in, Indicates will repeat The vector obtained this time;

[0090] In some specific implementations, based on an approximate discrete optimization mechanism, adjusting the device nodes and hyperedges in the first hypergraph structure to obtain the second hypergraph structure further includes:

[0091] Define the hidden state of device nodes and the hidden state of hyperedges. Specifically, define the hidden state of nodes. Hidden state representation Super-edge The hidden state is represented as ;

[0092] Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge, where the node... and super edge The formula for calculating the similarity between them is expressed as follows: ;

[0093] Based on the aforementioned similarity, the increment matrix and mask matrix of the hyperedge are determined. Specifically, the increment matrix of the hyperedge is calculated using the aforementioned similarity. and mask matrix Its calculation formula includes:

[0094]

[0095]

[0096] Based on the hyperedge mask, the hyperedge increment matrix, and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain the second hypergraph structure. Specifically, using several matrices obtained from hyperedge optimization and node optimization, the hypergraph structure optimization result learned from the hypergraph structure learning is represented as follows: ,in, Representing element-level multiplication, this application uses the first hypergraph structure... Further optimization ,in, It can only roughly depict a hypergraph structure, while This allows for further adjustments to the number of hyperedges in the hypergraph and the number of nodes contained within each hyperedge. Specifically, during the initial optimization iteration, [the following will be done]. and The nodes and hyperedges are fixed as matrices of all zeros and all one values ​​respectively. In subsequent iterations, the hidden states of the nodes and hyperedges obtained in the previous iteration are used. and calculate and .

[0097] In the above implementation, by introducing a more refined graph structure learning technique to adjust the hypergraph structure among multiple variables, namely by deleting unnecessary hyperedges, removing noisy nodes from hyperedges, and adding valid nodes to hyperedges, the hypergraph structure can be adjusted more finely. This helps to characterize the heterogeneity between nodes in real CPS systems based on the hypergraph structure and overcomes the limitations of existing methods that use KNN to construct graph structures.

[0098] S103. Model the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device node.

[0099] It should be noted that, as Figure 4 As shown, the spatiotemporal hypergraph neural network model includes a hypergraph neural network for handling spatial relationships and a Transformer (attention mechanism model) for handling temporal relationships. This application uses hypergraph structure learning to characterize the spatial interaction relationships between nodes and uses the Transformer to characterize the temporal relationships within each node, forming a spatiotemporal hypergraph neural network model (STHGNN) to encode and represent existing multivariate time series and predict their future states. A multivariate time series refers to a sequence formed by multiple data variables changing over time. Multivariate time series have characteristics in both time and space dimensions, and this data variable can be acquired through sensors on device nodes. Indicates the window length is MTS (Multivariate Time Series), where, Representative moment Time system The state values ​​of each node. This represents the number of nodes (devices or sensors), which is the second hypergraph structure obtained from the steps above. By inputting multivariate time series data into the spatiotemporal hypergraph neural network model STHGNN, the output can be obtained for... System state prediction at time 1 .

[0100] In some specific implementations, modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model includes:

[0101] Based on the learnable embedding vectors of the device nodes and the multivariate time series, the feature matrix of the device nodes is determined. Specifically, for the time window... any time within The features of all nodes can be represented as a node feature matrix. The hypergraph structure between nodes is ;

[0102] A hypergraph neural network is determined to represent the hypergraph data, which is a time series composed of the device node feature matrix and the second hypergraph structure. The hypergraph neural network is an HGNN, and time series data is determined based on the HGNN. Spatial relation learning, namely hypergraph data composed of the feature matrix and hypergraph structure;

[0103] An attention mechanism model for processing the time series is determined, wherein the attention mechanism model is a Transformer, which is typically used to process sequential data, such as text or time series. A standard Transformer typically includes position encoding, multi-head attention, residual connections, feedforward neural networks, and layer normalization components. This application utilizes a standard Transformer model to model the time series.

[0104] The spatiotemporal hypergraph neural network model is generated by combining the hypergraph neural network and the attention mechanism model.

[0105] In some specific implementations, methods for representing hypergraph data based on hypergraph neural networks include:

[0106] Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges;

[0107] Aggregate the features of the hyperedge to the corresponding device node;

[0108] Repeatedly iterate the aggregation operation to obtain the hidden state of nodes and hyperedges at the target time within the target time window.

[0109] Specifically, a general two-step hypergraph neural network (HGNN) is used to complete the time step. In spatial relation learning, the first step of HGNN is to aggregate the features of nodes to hyperedges, the expression of which is: ,in They represent the first Nodes during layer iteration and super edge The hidden state, and the initialization method for both are as follows: The second step of HGNN is to aggregate the features of the hyperedges to the nodes, i.e. By performing the above two steps Through iterative steps, we can eventually obtain the result at time [time]. Hidden states of nodes and hyperedges ,in, This indicates a predefined dimension. This represents the preset number of layers. The optimal value is typically obtained through a validation set, with the preferred value being: The type of hypergraph neural network can be set according to actual needs, and no specific limitation is made here. That is, the above Update function and Aggregate function can be instantiated into a specific neural network. It is preferred to use DeepSet or SetTransformer architecture to instantiate the above functions.

[0110] Furthermore, the hidden states of nodes and hyperedges at multiple target times within the target time window are aggregated to obtain a time series of the fused spatial relationships of the second hypergraph structure within the target time window, i.e., a time series of fused spatial relationships. The target time window value can be set according to actual needs; here it is [value missing]. Specifically, through the time window Repeating the above operation at all times within the hypergraph space yields a series of time series of hidden states of nodes and hyperedges that incorporate the relationships within the hypergraph space. and For example, for a node Its time series can be represented as For super-edge Its time series can be represented as .

[0111] In some specific implementations, methods for processing time series based on attention mechanism models include:

[0112] Based on an attention mechanism, the time series is encoded to obtain the hidden states of the target node and target hyperedge. The spatial relationships of the multivariate time series have already been integrated into the hidden states through the aforementioned node and hyperedge time series. Next, the standard Transformer model is used to model the time series to characterize the temporal relationships within it, and a TransEncoder is used to represent the entire set of components. Specifically, the TransEncoder is used to encode the time series of nodes and hyperedges, represented as follows: , Here we get and It will be further used for the optimization of hypergraph structure nodes in step S102 above.

[0113] In the above implementation, the Transformer model is used to model the time series information of each node, and the hypergraph neural network HGNN is combined to generate the spatiotemporal hypergraph neural network STHGNN to represent and predict multivariate time series data, thereby better completing the anomaly detection task and improving the accuracy and efficiency of anomaly detection.

[0114] S104. Combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model is generated. The fully connected neural network is used to predict the state of the device node at the next moment based on the output value of the spatiotemporal hypergraph neural network model.

[0115] It should be noted that a fully connected neural network refers to... .

[0116] Specifically, each device node is obtained based on the spatiotemporal hypergraph neural network model. Hidden state Through a simple fully connected neural network To predict variables exist The state value at time t, i.e., calculation: Based on this, it is possible to utilize variable time series. Predicting the next moment of the system The task is to determine the state of the problem. Therefore, by combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model can be generated.

[0117] In the above implementation, using a model based on a spatiotemporal hypergraph neural network to encode existing time series and predict future system states can effectively characterize and utilize the temporal and spatial dependencies of data, help to better fit training data, provide more accurate predictions, and provide assurance for subsequent anomaly detection.

[0118] S105. Using the target dataset and the mean squared error loss function, train and validate the initial anomaly detection model to generate the target anomaly detection model.

[0119] It should be noted that the target dataset refers to the multivariate time series data collected by different sensors during the normal operation of the target CPS system. This means that there are no outliers in the training and validation data, and the system utilizes normal data through unsupervised learning mechanisms. The anomaly detection model constructed in the above steps will be trained and validated offline.

[0120] In some specific implementations, the initial anomaly detection model is trained and validated using the target dataset and the mean squared error loss function to generate the target anomaly detection model, including:

[0121] The target dataset is divided into a training set and a validation set. The division ratio can be set according to actual needs, such as 80% for the training set and 20% for the validation set. The target dataset is then divided into segments with a window length of [missing information]. Window data To predict the state of the last step in the window data. The window length can be set according to actual needs, with the preferred value being [value missing]. ;

[0122] The initial anomaly detection model is trained and validated based on the adaptive optimizer, the training set, and the validation set. The loss value corresponding to the validation set is determined based on the mean squared error loss function. The training method employs gradient descent, and the model is optimized using an adaptive optimizer such as Adam. The mean squared error loss function includes:

[0123]

[0124] in, Indicates the time series length of all training data;

[0125] Based on the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and the target anomaly detection model is output. The initial learning rate, the learning rate adjustment value, and the preset standard can be set according to actual needs. For example, the learning rate is set to 0.001, and the learning rate is adjusted according to the loss value of the validation set, with each adjustment factor being 0.5, until the learning rate reaches 0.00001. This application uses an early stopping mechanism to avoid model overfitting, and the patience value of the early stopping mechanism is set to 10.

[0126] In the above implementation, the anomaly detection model is trained without relying on any labeled anomaly data, which effectively avoids the problems of difficulty in obtaining real anomaly data and high labeling costs, reduces the implementation difficulty, and improves the applicability of the anomaly detection model.

[0127] The above-described multivariate time series anomaly detection method includes: initializing learnable embedding vectors for device nodes and determining hyperedges of a first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; adjusting the device nodes and hyperedges in the first hypergraph structure based on an approximate discrete optimization mechanism to obtain a second hypergraph structure; modeling the learnable embedding vectors, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes; and combining the spatiotemporal hypergraph neural network model and a fully connected neural network to generate an initial anomaly detection model. The fully connected neural network is used to predict the state of the device node at the next time step based on the output value of the spatiotemporal hypergraph neural network model. The initial anomaly detection model is trained and validated using the target dataset and the mean squared error loss function to generate the target anomaly detection model. This application introduces the hypergraph structure into the anomaly detection problem of multivariate time series, which can better characterize the complex high-order interaction relationships between different device nodes in a real system, and is beneficial for fitting the system's evolution. By modeling multivariate time series using a spatiotemporal hypergraph neural network, spatial relationships and temporal information are modeled, thereby better completing the multivariate time series anomaly detection task and improving detection efficiency and accuracy.

[0128] It should be understood that, although Figures 3-4 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 3-4 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0129] In one embodiment, such as Figure 5 As shown, a multivariate time series anomaly detection method is provided, which is applied to... Figure 2 Taking the terminal in the example, the explanation includes the following steps:

[0130] S201. Obtain the multivariate time series of device nodes in the system to be tested.

[0131] It should be noted that the system to be detected refers to a CPS system, but it can also be other cyber-physical systems that require anomaly detection. The multivariate time series data is acquired through sensors on the device nodes.

[0132] S202. Input the multivariate time series into the target anomaly detection model constructed by the above method, and output the predicted value.

[0133] It should be noted that after generating the target anomaly detection model, it can generally be deployed to the corresponding system to predict the system's state at the next moment based on the collected multivariate time series. The deployed system can include cyber-physical systems such as water resource monitoring systems, power grid monitoring systems, and server monitoring systems. The target anomaly detection model can predict the "normal" system state. When the actual system state deviates significantly from the predicted system state, the actual system state can be considered to be in an "abnormal" state. Based on this, abnormal situations in the system can be detected.

[0134] S203. Calculate the deviation between the predicted value and the true value, and determine the state of the system to be detected based on the comparison result between the first target deviation value and the first preset threshold.

[0135] It should be noted that the predicted value refers to the output value of the target anomaly detection model, the true value refers to the system state value obtained at the current time node, and the first preset threshold is the maximum error value corresponding to all times during the validation process of the anomaly detection model based on the validation set.

[0136] In some specific embodiments, after calculating the deviation between the predicted value and the actual value, the method further includes:

[0137] The deviation value is robustly normalized to obtain the deviation value of the first target node;

[0138] The deviation values ​​of multiple first target nodes at the same time point of the system are sorted. Based on the sorting results, the first target deviation value is determined, that is, the maximum value in the sorting results is selected and defined as the first target deviation value.

[0139] Specifically, through Represents a node At any moment The deviation between the predicted and the actual value is such that, since different sensors (nodes) may have very different characteristics, their deviation values ​​may also have very different scales. To prevent the deviation generated by any one sensor from excessively dominating the error values ​​of other sensors, a robust normalization is performed on the error value of each sensor, and the formula is as follows:

[0140]

[0141] in, and They represent The median and quartiles of the values ​​across the time scale are used in this application instead of the mean and standard deviation to improve the robustness of anomaly detection.

[0142] Furthermore, utilizing The maximum value of all node error values ​​at time 1 is used as the criterion for determining whether an error has occurred in the system at that time. .

[0143] In some specific implementations, determining the state of the system to be detected based on the comparison result between the first target deviation value and the first preset threshold includes:

[0144] Obtain the second target node deviation value obtained during the validation of the initial anomaly detection model. The second target node deviation value refers to the error value at all times during the validation of the anomaly detection model based on the validation set.

[0145] The deviation values ​​of the second target node corresponding to multiple time points within the target time window are sorted. Based on the sorting results, a first preset threshold is determined, which is the maximum error value reached at all times on the validation set. Defined as the first preset threshold;

[0146] In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system under test is in an abnormal state, that is... and When comparing, Greater than the threshold When, it means at the predicted time. When the system malfunctions, it means the system is in an abnormal state.

[0147] In the above implementation, the system state is predicted by constructing a target anomaly detection model, and the final state of the system is evaluated by combining the prediction results, the Lubang normalization mechanism and the threshold determined based on the validation set, which further improves the accuracy of multivariate time series anomaly detection.

[0148] In some specific embodiments, the method further includes:

[0149] Obtain robustly normalized error values ​​corresponding to sensors at multiple time points, and calculate the first sensing coefficients corresponding to multiple error values. The number of time points can be determined according to actual needs. The error value calculation formula is as follows:

[0150]

[0151] in, Represents the first perception coefficient. Indicates the number of error values. , They represent the first The and the first One error value;

[0152] In response to the detection of the first sensing coefficient When the value is less than a second preset threshold, a second sensing coefficient corresponding to the sensor is determined. The second preset threshold can be set according to actual needs. The calculation method for the second sensing coefficient includes:

[0153]

[0154] in, This represents the second perception coefficient. Indicates the deviation value;

[0155] In response to detecting that the second sensing coefficient is less than a third preset threshold, the device node corresponding to the sensor is marked, and in subsequent multivariate time series anomaly detection, the data is directly used. The calculated deviation values ​​are used for sorting to determine whether the system has an error at that moment. The third preset threshold can be determined according to actual needs.

[0156] In the above implementation, by marking the device nodes corresponding to the sensors, it is not necessary to robustly normalize the error values ​​of the sensors during subsequent anomaly detection, thereby improving detection efficiency while ensuring detection accuracy.

[0157] It should be understood that, although Figure 5 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 5 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0158] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a multivariate time series anomaly detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0159] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0161] S101. Initialize the learnable embedding vectors of the device nodes, and determine the hyperedges of the first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges.

[0162] S102. Based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure;

[0163] S103. Model the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device node;

[0164] S104. Combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model is generated. The fully connected neural network is used to predict the state of the device node at the next moment based on the output value of the spatiotemporal hypergraph neural network model.

[0165] S105. Using the target dataset and the mean squared error loss function, train and validate the initial anomaly detection model to generate the target anomaly detection model.

[0166] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0167] Using a vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated.

[0168] The similarity scores are sorted from largest to smallest to obtain the sorting results, and the target number of device nodes are selected based on the sorting results.

[0169] With the device node corresponding to the target learnable embedding vector as the center, construct a hyperedge containing the number of device nodes of the target;

[0170] The first hypergraph structure is generated based on device nodes and hyperedges.

[0171] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0172] Initialize the learnable sampling parameters of the hyperedge;

[0173] Based on the discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0174] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0175] Define the hidden state of the device node and the hidden state of the superedge;

[0176] Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge;

[0177] Based on the similarity, determine the increment matrix and mask matrix of the hyperedge;

[0178] Based on the hyperedge mask, the hyperedge increment matrix, and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain the second hypergraph structure.

[0179] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0180] Based on the learnable embedding vectors of the device nodes and the multivariate time series, the feature matrix of the device nodes is determined;

[0181] A hypergraph neural network is determined to characterize the hypergraph data, wherein the hypergraph data is a time series composed of the device node feature matrix and the second hypergraph structure;

[0182] Determine an attention mechanism model for processing the time series;

[0183] The spatiotemporal hypergraph neural network model is generated by combining the hypergraph neural network and the attention mechanism model.

[0184] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0185] Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges;

[0186] Aggregate the features of the hyperedge to the corresponding device node;

[0187] Repeatedly iterate the aggregation operation to obtain the hidden state of nodes and hyperedges at the target time within the target time window.

[0188] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0189] The hidden states of nodes and hyperedges at multiple target times within the target time window are aggregated to obtain the time series of the spatial relationships of nodes and hyperedges in the second hypergraph structure within the target time window.

[0190] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0191] Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0192] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0193] The target dataset is divided into a training set and a validation set;

[0194] The initial anomaly detection model is trained and validated based on the adaptive optimizer, the training set, and the validation set, and the loss value corresponding to the validation set is determined based on the mean squared error loss function.

[0195] Based on the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and then the target anomaly detection model is output.

[0196] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0197] S201. Obtain the multivariate time series of device nodes in the system to be tested;

[0198] S202. Input the multivariate time series into the target anomaly detection model constructed by the above method, and output the predicted value;

[0199] S203. Calculate the deviation between the predicted value and the true value, and determine the state of the system to be detected based on the comparison result between the first target deviation value and the first preset threshold.

[0200] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0201] The deviation value is robustly normalized to obtain the deviation value of the first target node;

[0202] The deviation values ​​of multiple first target nodes at the same time point of the system are sorted, and the first target deviation value is determined based on the sorting results.

[0203] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0204] Obtain the deviation value of the second target node obtained during the validation of the initial anomaly detection model;

[0205] The deviation values ​​of the second target node corresponding to multiple time nodes within the target time window are sorted, and the first preset threshold is determined based on the sorting results.

[0206] When the first target deviation value is detected to be greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0207] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0208] S101. Initialize the learnable embedding vectors of the device nodes, and determine the hyperedges of the first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges.

[0209] S102. Based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure;

[0210] S103. Model the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device node;

[0211] S104. Combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model is generated. The fully connected neural network is used to predict the state of the device node at the next moment based on the output value of the spatiotemporal hypergraph neural network model.

[0212] S105. Using the target dataset and the mean squared error loss function, train and validate the initial anomaly detection model to generate the target anomaly detection model.

[0213] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0214] Using a vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated.

[0215] The similarity scores are sorted from largest to smallest to obtain the sorting results, and the target number of device nodes are selected based on the sorting results.

[0216] With the device node corresponding to the target learnable embedding vector as the center, construct a hyperedge containing the number of device nodes of the target;

[0217] The first hypergraph structure is generated based on device nodes and hyperedges.

[0218] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0219] Initialize the learnable sampling parameters of the hyperedge;

[0220] Based on the discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0221] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0222] Define the hidden state of the device node and the hidden state of the superedge;

[0223] Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge;

[0224] Based on the similarity, determine the increment matrix and mask matrix of the hyperedge;

[0225] Based on the hyperedge mask, the hyperedge increment matrix, and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain the second hypergraph structure.

[0226] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0227] Based on the learnable embedding vectors of the device nodes and the multivariate time series, the feature matrix of the device nodes is determined;

[0228] A hypergraph neural network is determined to characterize the hypergraph data, wherein the hypergraph data is a time series composed of the device node feature matrix and the second hypergraph structure;

[0229] Determine an attention mechanism model for processing the time series;

[0230] The spatiotemporal hypergraph neural network model is generated by combining the hypergraph neural network and the attention mechanism model.

[0231] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0232] Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges;

[0233] Aggregate the features of the hyperedge to the corresponding device node;

[0234] Repeatedly iterate the aggregation operation to obtain the hidden state of nodes and hyperedges at the target time within the target time window.

[0235] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0236] The hidden states of nodes and hyperedges at multiple target times within the target time window are aggregated to obtain the time series of the spatial relationships of nodes and hyperedges in the second hypergraph structure within the target time window.

[0237] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0238] Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0239] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0240] The target dataset is divided into a training set and a validation set;

[0241] The initial anomaly detection model is trained and validated based on the adaptive optimizer, the training set, and the validation set, and the loss value corresponding to the validation set is determined based on the mean squared error loss function.

[0242] Based on the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and then the target anomaly detection model is output.

[0243] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0244] S201. Obtain the multivariate time series of device nodes in the system to be tested;

[0245] S202. Input the multivariate time series into the target anomaly detection model constructed by the above method, and output the predicted value;

[0246] S203. Calculate the deviation between the predicted value and the true value, and determine the state of the system to be detected based on the comparison result between the first target deviation value and the first preset threshold.

[0247] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0248] The deviation value is robustly normalized to obtain the deviation value of the first target node;

[0249] The deviation values ​​of multiple first target nodes at the same time point of the system are sorted, and the first target deviation value is determined based on the sorting results.

[0250] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0251] Obtain the deviation value of the second target node obtained during the validation of the initial anomaly detection model;

[0252] The deviation values ​​of the second target node corresponding to multiple time nodes within the target time window are sorted, and the first preset threshold is determined based on the sorting results.

[0253] When the first target deviation value is detected to be greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0254] In one embodiment, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, performs the following steps:

[0255] S101. Initialize the learnable embedding vectors of the device nodes, and determine the hyperedges of the first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges.

[0256] S102. Based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure;

[0257] S103. Model the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device node;

[0258] S104. Combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model is generated. The fully connected neural network is used to predict the state of the device node at the next moment based on the output value of the spatiotemporal hypergraph neural network model.

[0259] S105. Using the target dataset and the mean squared error loss function, train and validate the initial anomaly detection model to generate the target anomaly detection model.

[0260] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0261] Using a vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated.

[0262] The similarity scores are sorted from largest to smallest to obtain the sorting results, and the target number of device nodes are selected based on the sorting results.

[0263] With the device node corresponding to the target learnable embedding vector as the center, construct a hyperedge containing the number of device nodes of the target;

[0264] The first hypergraph structure is generated based on device nodes and hyperedges.

[0265] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0266] Initialize the learnable sampling parameters of the hyperedge;

[0267] Based on the discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0268] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0269] Define the hidden state of the device node and the hidden state of the superedge;

[0270] Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge;

[0271] Based on the similarity, determine the increment matrix and mask matrix of the hyperedge;

[0272] Based on the hyperedge mask, the hyperedge increment matrix, and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain the second hypergraph structure.

[0273] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0274] Based on the learnable embedding vectors of the device nodes and the multivariate time series, the feature matrix of the device nodes is determined;

[0275] A hypergraph neural network is determined to characterize the hypergraph data, wherein the hypergraph data is a time series composed of the device node feature matrix and the second hypergraph structure;

[0276] Determine an attention mechanism model for processing the time series;

[0277] The spatiotemporal hypergraph neural network model is generated by combining the hypergraph neural network and the attention mechanism model.

[0278] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0279] Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges;

[0280] Aggregate the features of the hyperedge to the corresponding device node;

[0281] Repeatedly iterate the aggregation operation to obtain the hidden state of nodes and hyperedges at the target time within the target time window.

[0282] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0283] The hidden states of nodes and hyperedges at multiple target times within the target time window are aggregated to obtain the time series of the spatial relationships of nodes and hyperedges in the second hypergraph structure within the target time window.

[0284] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0285] Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0286] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0287] The target dataset is divided into a training set and a validation set;

[0288] The initial anomaly detection model is trained and validated based on the adaptive optimizer, the training set, and the validation set, and the loss value corresponding to the validation set is determined based on the mean squared error loss function.

[0289] Based on the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and then the target anomaly detection model is output.

[0290] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0291] S201. Obtain the multivariate time series of device nodes in the system to be tested;

[0292] S202. Input the multivariate time series into the target anomaly detection model constructed by the above method, and output the predicted value;

[0293] S203. Calculate the deviation between the predicted value and the true value, and determine the state of the system to be detected based on the comparison result between the first target deviation value and the first preset threshold.

[0294] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0295] The deviation value is robustly normalized to obtain the deviation value of the first target node;

[0296] The deviation values ​​of multiple first target nodes at the same time point of the system are sorted, and the first target deviation value is determined based on the sorting results.

[0297] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0298] Obtain the deviation value of the second target node obtained during the validation of the initial anomaly detection model;

[0299] The deviation values ​​of the second target node corresponding to multiple time nodes within the target time window are sorted, and the first preset threshold is determined based on the sorting results.

[0300] When the first target deviation value is detected to be greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0301] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0302] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0303] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A method for detecting abnormal states in a cyber-physical system, characterized in that, The method includes: A multivariate time series of device nodes in a system under test is obtained. The system under test is a cyber-physical system (CPS), and the device nodes are devices within the CPS or sensors on those devices. A multivariate time series is a sequence of multiple data variables changing over time, possessing characteristics in both time and space. Data variables are acquired through sensors on the device nodes. Indicates the window length is The multivariate time series, where, Representative moment Time system The status values ​​of each device node. Indicates the number of device nodes; The multivariate time series is input into the target anomaly detection model, and the predicted value is output, that is, the predicted value of the cyber-physical system state at time t. The deviation between the predicted value and the actual value is calculated, and the state of the system to be detected is determined based on the comparison between the first target deviation value and the first preset threshold. This is achieved through... Represents device node At any moment The deviation between the predicted value and the actual value is used to obtain the first target deviation value after robust normalization of the deviation value. It also includes: obtaining robustly normalized error values ​​corresponding to sensors at multiple time points, calculating the first sensing coefficients corresponding to multiple error values, and the error value calculation formula is: ； in, The first perception coefficient, The number of error values. , The first The and the first One error value; If the first perception coefficient If the value is less than a second preset threshold, the second sensing coefficient corresponding to the sensor is determined, and its calculation method includes: ； in, The second perception coefficient, This is the deviation value; If the second sensing coefficient is less than the third preset threshold, the device node corresponding to the sensor is marked, and the deviation value of the device node is defined as the first target deviation value. Determining the state of the system to be detected based on the comparison between the first target deviation value and the first preset threshold includes: Obtain the deviation value of the second target node obtained during the validation of the initial anomaly detection model; The deviation values ​​of the second target node corresponding to multiple time nodes within the target time window are sorted, and the first preset threshold is determined based on the sorting results. When the first target deviation value is detected to be greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state; The method for generating the target anomaly detection model includes: initializing learnable embedding vectors of device nodes and determining hyperedges of a first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; adjusting the device nodes and hyperedges in the first hypergraph structure based on an approximate discrete optimization mechanism to obtain a second hypergraph structure; modeling the learnable embedding vectors, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired by sensors on the device nodes; combining the spatiotemporal hypergraph neural network model and a fully connected neural network to generate an initial anomaly detection model, wherein the fully connected neural network is used to predict the state of the device nodes at the next time step based on the output value of the spatiotemporal hypergraph neural network model; training and validating the initial anomaly detection model using a target dataset and a mean squared error loss function to generate a target anomaly detection model; after generating the target anomaly detection model, deploying the target anomaly detection model to the corresponding cyber-physical system to detect the state of the cyber-physical system, wherein the learnable embedding vectors are obtained by randomly initializing the device nodes, and the hyperedges consist of multiple device nodes.

2. The cyber-physical system abnormal state detection method according to claim 1, characterized in that, Based on the similarity between learnable embedding vectors, the hyperedges of the first hypergraph structure are determined to include: Using a vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated. The similarity scores are sorted from largest to smallest to obtain the sorting results, and the target number of device nodes are selected based on the sorting results. With the device node corresponding to the target learnable embedding vector as the center, construct a hyperedge containing the number of device nodes of the target; The first hypergraph structure is generated based on device nodes and hyperedges.

3. The method for detecting abnormal states in a cyber-physical system according to claim 1, characterized in that, Based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure, which includes: Initialize the learnable sampling parameters of the hyperedge; Based on the discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

4. The cyber-physical system abnormal state detection method according to claim 3, characterized in that, Based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively, resulting in a second hypergraph structure that further includes: Define the hidden state of the device node and the hidden state of the superedge; Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge; Based on the similarity, determine the increment matrix and mask matrix of the hyperedge; Based on the hyperedge mask, the hyperedge increment matrix, and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain the second hypergraph structure.

5. The method for detecting abnormal states in a cyber-physical system according to claim 1, characterized in that, Modeling the learnable embedding vectors, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model includes: Based on the learnable embedding vectors of the device nodes and the multivariate time series, the feature matrix of the device nodes is determined; A hypergraph neural network is determined to characterize the hypergraph data, wherein the hypergraph data is a time series composed of the device node feature matrix and the second hypergraph structure; Determine an attention mechanism model for processing the time series; The spatiotemporal hypergraph neural network model is generated by combining the hypergraph neural network and the attention mechanism model.

6. The method for detecting abnormal states in a cyber-physical system according to claim 5, characterized in that, Methods for representing hypergraph data based on hypergraph neural networks include: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device node; Repeatedly iterate the aggregation operation to obtain the hidden state of nodes and hyperedges at the target time within the target time window.

7. The method for detecting abnormal states in a cyber-physical system according to claim 6, characterized in that, The method further includes: The hidden states of nodes and hyperedges at multiple target times within the target time window are aggregated to obtain the time series of the spatial relationships of nodes and hyperedges in the second hypergraph structure within the target time window.

8. The method for detecting abnormal states in a cyber-physical system according to claim 5, characterized in that, Methods for processing time series data based on attention mechanisms include: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

9. The method for detecting abnormal states in a cyber-physical system according to claim 1, characterized in that, The initial anomaly detection model is trained and validated using the target dataset and the mean squared error loss function to generate the target anomaly detection model, including: The target dataset is divided into a training set and a validation set; The initial anomaly detection model is trained and validated based on the adaptive optimizer, the training set, and the validation set, and the loss value corresponding to the validation set is determined based on the mean squared error loss function. Based on the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and then the target anomaly detection model is output.

10. The method for detecting abnormal states in a cyber-physical system according to claim 1, characterized in that, After calculating the deviation between the predicted value and the actual value, the method further includes: The deviation value is robustly normalized to obtain the deviation value of the first target node; The deviation values ​​of multiple first target nodes at the same time point of the system are sorted, and the first target deviation value is determined based on the sorting results.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 10.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

Images