Model construction method, multivariable time sequence anomaly detection method and related components

Abstract

The invention relates to a model construction method, a multivariable time sequence anomaly detection method and related components, and belongs to the technical field of system anomaly detection.The method comprises the steps that learnable embedding vectors of equipment nodes are initialized, and hyperedges of a first hypergraph structure are determined based on the similarity between the learnable embedding vectors; based on an approximate discrete optimization mechanism, respectively adjusting equipment nodes and hyperedges in the first hypergraph structure to obtain a second hypergraph structure; modeling the learnable embedded vector, the second hypergraph structure and the multivariable time sequence to generate a space-time hypergraph neural network model; generating an initial anomaly detection model in combination with the space-time hypergraph neural network model and the full-connection neural network; and training and verifying the initial anomaly detection model by using the target data set and the mean square error loss function to generate a target anomaly detection model. According to the invention, the detection efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of system anomaly detection, and in particular to a model building method, a multivariate time series anomaly detection method and related components. Background Art

[0002] With the rapid popularization of cyber-physical systems (CPS) such as industrial systems, data centers, the Internet of Things and the Internet of Vehicles, the monitoring and protection of CPS systems is becoming increasingly important, especially for key application infrastructures such as power grid systems, water resources monitoring systems and transportation systems.

[0003] The above-mentioned systems are generally equipped with many different (heterogeneous) sensors to detect abnormal conditions of the system. For example, in a water resource system, there are sensors for detecting water level, flow, water quality, and temperature. The data obtained from these monitoring devices or sensors will form complex multivariate time series data (MTS). It is a difficult task to automatically detect abnormal conditions of the system from these complex series data. The difficulties lie in: (1) Different time series may be collected by different types of sensors, and the changes between sensors may be correlated in time and space, making it difficult to analyze the complex nonlinear relationships using simple methods; (2) Some systems may have hundreds or thousands of CPS subsystems, each of which records a number of time series data, making the final total amount of data exceed the limit of manual management; (3) It is difficult to collect a large amount of system status when abnormal information occurs, that is, the system anomalies must be detected in an unsupervised manner. Summary of the invention

[0004] Based on this, it is necessary to provide a model construction method, a multivariate time series anomaly detection method and related components that can effectively mine the complex interactive relationships within multivariate time series to better complete the anomaly detection task in response to the above technical problems.

[0005] In a first aspect, a multivariate time series anomaly detection method is provided, the method comprising: Initializing learnable embedding vectors of device nodes, and determining hyperedges of a first hypergraph structure based on similarities between learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; Based on the approximate discrete optimization mechanism, the device nodes and the hyperedges in the first hypergraph structure are adjusted respectively to obtain a second hypergraph structure; Modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes; Combining the spatiotemporal hypergraph neural network model and the fully connected neural network to generate an initial anomaly detection model, wherein the fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model; The initial anomaly detection model is trained and verified using the target data set and the mean square error loss function to generate a target anomaly detection model.

[0006] Optionally, determining the hyperedge of the first hypergraph structure based on the similarity between the learnable embedding vectors includes: Using the vector similarity function, calculate the similarity between the target learnable embedding vector and other learnable embedding vectors; Sorting the similarities in descending order to obtain a sorting result, and selecting a target number of device nodes based on the sorting result; Taking the device node corresponding to the target learnable embedding vector as the center, constructing a hyperedge including the target number of device nodes; Based on the device nodes and the hyperedges, the first hypergraph structure is generated.

[0007] Optionally, based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain a second hypergraph structure including: Initialize the learnable sampling parameters of the hyperedge; Based on a discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0008] Optionally, based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain the second hypergraph structure, and the second hypergraph structure further includes: Define the hidden state of the device node and the hidden state of the hyperedge; Calculating the similarity between the hidden state of the device node and the hidden state of the hyperedge; Determining an increment matrix and a mask matrix of the hyperedge according to the similarity; Based on the hyperedge mask, the hyperedge increment matrix and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain a second hypergraph structure.

[0009] Optionally, modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model includes: Determining a device node feature matrix based on the learnable embedding vectors of the device nodes and the multivariate time series; Determine a hypergraph neural network for representing hypergraph data, wherein the hypergraph data is a time series consisting of the device node feature matrix and the second hypergraph structure; determining an attention mechanism model for processing the time series; The hypergraph neural network and the attention mechanism model are combined to generate the spatiotemporal hypergraph neural network model.

[0010] Optionally, the method for representing hypergraph data based on the hypergraph neural network includes: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device nodes; Repeat the iterative aggregation operation to obtain the hidden states of nodes and hyperedges at the target time within the target time window.

[0011] Optionally, the method further includes: The hidden states of nodes and hyperedges at multiple target moments in the target time window are aggregated to obtain a time series of nodes and hyperedges that fuse the spatial relationships of the second hypergraph structure in the target time window.

[0012] Optional methods for processing time series based on attention mechanism models include: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0013] Optionally, using the target data set and the mean square error loss function to train and verify the initial anomaly detection model to generate a target anomaly detection model includes: Dividing the target data set into a training set and a validation set; Based on the adaptive optimizer, the training set and the validation set, the initial anomaly detection model is trained and validated, and based on the mean square error loss function, a loss value corresponding to the validation set is determined; According to the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and the target anomaly detection model is output.

[0014] In a second aspect, a multivariate time series anomaly detection method is provided, the method comprising: Obtain multivariate time series of device nodes in the system to be detected; Input the multivariate time series into the target anomaly detection model constructed by the above method, and output the predicted value; The deviation value between the predicted value and the true value is calculated, and based on the comparison result between the first target deviation value and the first preset threshold value, the state of the system to be detected is determined.

[0015] Optionally, after calculating the deviation value between the predicted value and the true value, the method further includes: Performing robust normalization processing on the deviation value to obtain a first target node deviation value; Sorting multiple first target node deviation values ​​of the system at the same time node, and determining the first target deviation value according to the sorting result.

[0016] Optionally, determining the state of the system to be detected based on a comparison result between the first target deviation value and a first preset threshold value includes: Obtaining a second target node deviation value obtained in the process of verifying the initial anomaly detection model; Sorting the second target node deviation values ​​corresponding to multiple time nodes in the target time window, and determining a first preset threshold value according to the sorting result; In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the model building method provided in the first aspect and / or the multivariate time series anomaly detection method provided in the second aspect are implemented.

[0018] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the model building method provided in the first aspect and / or the multivariate time series anomaly detection method provided in the second aspect are implemented.

[0019] In a fifth aspect, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements the model building method provided in the first aspect and / or the multivariate time series anomaly detection method provided in the second method.

[0020] The above-mentioned model construction method, multivariate time series anomaly detection method and related components, by introducing the hypergraph structure into the anomaly detection problem of multivariate time series, can better characterize the complex high-order interaction relationship between different device nodes in the real system, which is conducive to fitting the evolution of the system. By modeling multivariate time series through spatiotemporal hypergraph neural network, spatial relationships and timing information are modeled, which can better complete the multivariate time series anomaly detection task and improve detection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 (a)-1(b) are schematic diagrams of a hypergraph and a pairwise interaction simple graph in one embodiment; Figure 2 1. is an application environment diagram of a multivariate time series anomaly detection method in one embodiment; Figure 3 A schematic diagram of a flow chart of a model building method in one embodiment; Figure 4 Another schematic diagram of a flow chart of a model building method in one embodiment; Figure 5 1 is a flow chart of a multivariate time series anomaly detection method according to an embodiment; Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It should be understood that in the description of the present application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like throughout the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as including but not limited to.

[0024] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0025] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present application. They are only for the convenience of describing the method of the present application, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0026] According to the background technology, it is difficult to automatically detect the abnormal state of the system from complex sequence data. To solve this problem, the existing technology captures the complex relationship between multivariate time series through graphs. The existing solution 1 randomly initializes a learnable embedding vector for each time series, and then uses KNN (K-NearestNeighbor, K nearest neighbor classification algorithm) to construct a graph structure based on the embedding vector. Then, based on the graph structure, the graph attention neural network is used to fuse the node's time series information and the embedding vector to predict the future state of each node, thereby completing the prediction-based anomaly detection goal. The subsequent specific process includes using the embedding based on KNN to generate a graph structure of different nodes (Sensors), using The graph structure, time series data and embedded vectors are used to complete the prediction of the time series, and then the prediction-based anomaly detection is completed. The existing scheme 2 has made corresponding improvements in each step based on the above scheme 1, including: 1. Adding learnable parameters in the process of embedding vectors to generate graph structures; 2. Using the time convolution network TGN to encode the time series of each node; 3. Combining the graph structure and TGN, a spatiotemporal graph convolution network STGNN is used as a whole to represent the existing time series and output the prediction of the time series; 4. When calculating the difference between the predicted value and the observed value, PCA technology (principal component analysis technology) is used instead of a simple absolute value distance; However, the above methods have the following problems: 1. They all assume that nodes directly interact in pairs (pairwise interaction), so a graph is used to model the relationship between nodes. However, this prior assumption is unreasonable. For example, Figure 1 (a) and Figure 1 (b) The hypergraph and the simple graph of pairwise interactions are shown respectively. In the simple graph of pairwise interactions, each edge can only connect 2 nodes, and a hyperedge in the hypergraph can contain any number of nodes. Specifically, in the water resources monitoring system, Four sensors monitor the condition of a reservoir together, and Three sensors jointly monitor the situation of a river. The first four sensors will be affected by the high-order factor "reservoir", while the last three sensors will be affected by the high-order factor "river", which will cause high-order correlation between sensors. It cannot be described by pairwise interaction. In the above scheme, this correlation is expressed by The fully connected graph and However, a more reasonable representation method is to use a hypergraph to represent Put it in a hyperedge representing a "reservoir", 1. The existing methods are too rough in constructing graph structures and cannot flexibly adjust the graph structures. They can only rely on KNN to connect each node to its K nearest neighbors to construct the graph. This graph construction method cannot capture and characterize the relationship between multivariate time series to the greatest extent. For example, some nodes are completely independent of other nodes. At this time, connecting them to K neighbor nodes will bring noise to the prediction of the node; 2. The existing time series of each node is very important for state prediction, while the existing solution 1 directly ignores the time series information, and the existing solution 2 only uses TCN (Temporal Convolutional Networks, a convolutional neural network architecture designed specifically for sequence modeling) to capture the time series information, and cannot effectively detect abnormal results based on the time series information.

[0027] To solve the above technical problems, the present application provides a model construction method, a multivariate time series anomaly detection method and related components, using a hypergraph to represent the high-order dependencies between nodes, and dynamically adjusting the hypergraph structure to remove noise nodes from hyperedges and add valid nodes to hyperedges to obtain a more flexible and effective hypergraph structure, thereby providing a basis for accurate prediction of abnormal results, and further improving the accuracy and efficiency of multivariate time series anomaly detection by constructing a more effective timing model to capture the timing information of each node.

[0028] The model building method and multivariate time series anomaly detection method provided in this application can be applied to Figure 2 In the application environment shown, the terminal 102 communicates with the data processing platform set on the server 104 through the network, wherein the terminal 102 can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0029] In one embodiment, Figure 3 As shown, a model construction method is provided, which can be applied to Figure 2 The terminal in is used as an example to illustrate, including the following steps: S101, initializing learnable embedding vectors of device nodes, and determining hyperedges of a first hypergraph structure based on similarities between learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges.

[0030] It should be noted that a device node may refer to a device in a cyber-physical system (CPS) or a sensor on a device; a learnable embedding vector is obtained by randomly initializing the device node; a hyperedge is composed of multiple device nodes; and a hypergraph structure is composed of a set of nodes and a set of hyperedges. Each node may carry additional information, including node embedding information and time series information, which may be collectively referred to as node features.

[0031] In some specific implementations, determining a hyperedge of the first hypergraph structure based on the similarity between the learnable embedding vectors includes: Using the vector similarity function, the similarity between the target learnable embedding vector and other learnable embedding vectors is calculated. In the CPS system, the heterogeneity of nodes is obvious. Different nodes may represent different types of equipment and are in different working environments. In order to automatically capture the heterogeneity of nodes, this application uses learnable node embedding to adaptively learn the features of different nodes, and uses the node embedding for subsequent hypergraph structure learning. Specifically, for device nodes, this application randomly initializes The learnable node embedding vector is ,in , indicating a device node of dimensional embedding vector, which can be optimized by gradient Further, the present application uses a learnable hypergraph structure to capture high-order interactions between device nodes. The learning and optimization of the hypergraph structure needs to start from a rough hypergraph structure, which is called the initialization of the hypergraph structure. The hypergraph structure is initialized by the node embedding vector in the above steps to obtain the first hypergraph structure. Specifically, the vector similarity function is used to calculate the similarity of each node pair. The similarity between the target learnable embedding vector and other learnable embedding vectors, where the vector similarity function can be selected according to actual needs, such as cosine similarity, that is, calculating the cosine similarity between the embedding vectors of two nodes. When the cosine similarity is used to calculate the similarity, the expression of cosine similarity is ,in, Indicates similarity; The similarities are sorted in descending order to obtain a sorting result, and based on the sorting result, the target number of device nodes are selected, wherein the larger the similarity value, the higher the similarity. Therefore, the device nodes corresponding to the similarities of the top-ranked target number are selected to construct a hyperedge, and the target number You can choose according to actual needs; With the device node corresponding to the target learnable embedding vector as the center, a hyperedge containing the target number of device nodes is constructed. Specifically, when the device node corresponding to the target learnable embedding vector is When each node Build a hub containing Hyperedges , the hyperedge contains the node The most similar nodes, we can further get all the initialized hyperedge sets ; Based on the device nodes and the hyperedges, the first hypergraph structure is generated. Specifically, based on the device node set and the hyperedge set, the hypergraph structure can be initialized to generate the first hypergraph structure, wherein the number of device nodes and the number of hyperedges are , for convenience, we can use the incidence matrix To represent the hypergraph structure:

[0032] The first hypergraph structure is represented as .

[0033] In the above implementation, by introducing the hypergraph structure into the anomaly detection problem of multivariate time series, the complex high-order interaction relationship between different devices in the real system can be better characterized, which is conducive to fitting the evolution of the CPS system, and then obtaining more accurate system status prediction and anomaly detection results, and is conducive to operation and maintenance personnel to understand the system's operating mechanism and the root cause of system anomalies.

[0034] S102: Based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively to obtain a second hypergraph structure.

[0035] It should be noted that the approximate discrete optimization mechanism includes hyperedge sampling and node sampling, which are used to optimize and adjust device nodes and hyperedges. Hyperedge optimization is used to delete redundant hyperedges introduced during the hypergraph initialization process. Node optimization is used to delete redundant nodes in hyperedges and include hidden nodes that are not included in the hyperedges to further optimize the first hypergraph structure. Specifically, the first hypergraph structure initialized with TopK (finding the top k most important or highest-ranked elements from a set containing a large number of data items) cannot characterize the heterogeneity of each device node in a fine-grained manner. For example, for completely independent nodes (sensors), In addition, for some high-order interactions, such as weather factors, which will affect the status of more than k nodes, limiting the size of the hyperedge to k will impair the model's ability to capture such high-order interactions. Therefore, it is necessary to automatically remove nodes from inappropriate hyperedges through high-order structure learning, and put the nodes omitted by the TopK method in step S102 into the corresponding hyperedges. Based on this, the above goal is achieved through an approximate discrete optimization mechanism composed of hyperedge sampling and node sampling.

[0036] In some specific implementations, based on an approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively, and the second hypergraph structure is obtained, including: Initialize the learnable sampling parameters of the hyperedge. Specifically, initialize a learnable sampling parameter for each hyperedge in the first hypergraph structure. , expressed as , the parameter can be optimized through gradient optimization; Based on the discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask, wherein the discrete distribution sampling mechanism refers to the Gumbel sampling mechanism, that is, in order to ensure differentiability, Gumbel sampling is used to complete the get process, For each hyperedge The hyperedge mask of is expressed as:

[0037] in, , represents the sigmoid function, represents the temperature parameter; The mask of the hyperedge sampling of the first hypergraph structure is expressed as:

[0038] in, Indicates that repeat The vector obtained by this step; In some specific implementations, based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively, and the second hypergraph structure is obtained, further comprising: Define the hidden state of the device node and the hidden state of the hyperedge. Specifically, the node The hidden state representation of , super edge The hidden state is represented as ; Calculate the similarity between the hidden state of the device node and the hidden state of the hyperedge, where the node and super edge The similarity calculation formula between ; According to the similarity, the incremental matrix and mask matrix of the hyperedge are determined. Specifically, the incremental matrix of the hyperedge is calculated using the similarity. and the mask matrix , and its calculation formula includes:

[0039]

[0040] Based on the hyperedge mask, the hyperedge increment matrix and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain a second hypergraph structure. Specifically, the hypergraph structure optimization result of the hypergraph structure learning is expressed as ,in, Represents element-level multiplication. Based on this, the present application converts the first hypergraph structure into Further optimized to ,in, It can only roughly describe a hypergraph structure. The number of hyperedges in the hypergraph and the nodes contained in each hyperedge can be further adjusted. In the first iteration optimization, and They are fixed to all-0 and all-1 matrices respectively. In subsequent iterative optimization, the node and hyperedge hidden states obtained in the previous iteration are used and calculate and .

[0041] In the above implementation, a more detailed graph structure learning technology is introduced to adjust the hypergraph structure between multiple variables, that is, unnecessary hyperedges are deleted, noise nodes are deleted from the hyperedges, and real nodes are added to the hyperedges, so that the hypergraph structure can be adjusted in a more fine-grained manner, which helps to characterize the heterogeneity between nodes in the real CPS system based on the hypergraph structure, and overcomes the limitations of the existing method of using KNN to construct the graph structure.

[0042] S103. Modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes.

[0043] It should be noted that if Figure 4 As shown, the spatiotemporal hypergraph neural network model includes a hypergraph neural network for processing spatial relationships and a Transformer (attention mechanism model) for processing temporal relationships. The present application uses hypergraph structure learning to characterize the interaction relationship between nodes in the spatial dimension, and uses Transformer to characterize the temporal relationship within each node. The overall model constitutes a spatiotemporal hypergraph neural network model STHGNN, which is used to encode and characterize existing multivariate time series and predict their future states. A multivariate time series refers to a sequence formed by multiple data variables changing over time. A multivariate time series has the characteristics of two dimensions, time and space. The data variable can be obtained through sensors on device nodes, and can be obtained through Indicates that the window length is MTS (Multivariate Time Series), where Representative moments Time System The state value of each node, Represents the number of nodes (devices or sensors), that is, the second hypergraph structure obtained in the above steps , input the multivariate time series into the spatiotemporal hypergraph neural network model STHGNN, and output System status prediction at any moment .

[0044] In some specific implementations, modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model includes: Based on the learnable embedding vector of the device node and the multivariate time series, determine the device node feature matrix, specifically, for the time window Any time within , the features of all nodes can be expressed as the node feature matrix , the hypergraph structure between nodes is ; Determine a hypergraph neural network for representing hypergraph data, wherein the hypergraph data is a time series composed of the device node feature matrix and the second hypergraph structure, wherein the hypergraph neural network is HGNN, and the time series is completed based on the HGNN Spatial relationship learning, that is, the hypergraph data composed of the feature matrix and the hypergraph structure; Determine an attention mechanism model for processing the time series, wherein the attention mechanism model is a Transformer, which is usually used to process sequence data, such as text or time series. A standard Transformer usually includes position encoding, multi-head attention, residual connection, feedforward neural network and layer normalization components. This application uses a standard Transformer model to model time series; The hypergraph neural network and the attention mechanism model are combined to generate the spatiotemporal hypergraph neural network model.

[0045] In some specific embodiments, the method for representing hypergraph data based on a hypergraph neural network includes: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device nodes; Repeat the iterative aggregation operation to obtain the hidden states of nodes and hyperedges at the target time within the target time window.

[0046] Specifically, a general two-step hypergraph neural network HGNN is used to complete the time The first step of HGNN is to aggregate the features of nodes into hyperedges, which is expressed as ,in Respectively represent Layer iteration node and super edge The hidden state of both is initialized as ; The second step of HGNN is to aggregate the features of hyperedges to nodes, that is , by performing the above two steps After iteration of steps, we can finally get The hidden states of nodes and hyperedges ,in, represents a preset dimension, Indicates the preset number of layers. Generally, the best setting value is obtained through the validation set. The preferred value is: The type of the hypergraph neural network can be set according to actual needs and is not specifically limited here. That is, the above Update function and Aggregate function can be instantiated as a specific neural network, and it is preferred to use the DeepSet or SetTransformer architecture to instantiate the above functions.

[0047] Furthermore, the hidden states of nodes and hyperedges at multiple target moments in the target time window are aggregated to obtain a time series of nodes and hyperedges that fuse the spatial relationship of the second hypergraph structure in the target time window, that is, a time series of fused spatial relationships, wherein the target time window value can be set according to actual needs, and is here Specifically, by Repeating the above operation at all moments within can obtain a series of node and hyperedge hidden state time series that integrate the hypergraph spatial relationship and ; For example, for node , its time series can be expressed as , for the hyperedge , its time series can be expressed as .

[0048] In some specific implementations, the method for processing a time series based on an attention mechanism model includes: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target nodes and target hyperedges. The spatial relationship of the multivariate time series has been integrated into the hidden state through the above-mentioned node and hyperedge time series. Next, the standard Transformer model is used to model the time series to characterize the temporal relationship in the time series, and TransEncoder is used to represent the collection of the entire component; specifically, TransEncoder is used to encode the time series of nodes and hyperedges, which is expressed as , , here we get and It will be further used for the hypergraph structure node optimization in the above step S102.

[0049] In the above implementation, the Transformer model is used to model the time series information of each node, and combined with the hypergraph neural network HGNN, a spatiotemporal hypergraph neural network STHGNN is generated to characterize and predict multivariate time series data, so as to better complete the anomaly detection task and improve the accuracy and efficiency of anomaly detection.

[0050] S104. Generate an initial anomaly detection model by combining the spatiotemporal hypergraph neural network model and the fully connected neural network, wherein the fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model.

[0051] It should be noted that a fully connected neural network refers to .

[0052] Specifically, each device node obtained based on the spatiotemporal hypergraph neural network model The hidden state , through a simple fully connected neural network To predict variables exist The state value at the moment, that is, calculation: Based on this, we can use the variable time series Predict the next moment of the system Therefore, by combining the spatiotemporal hypergraph neural network model and the fully connected neural network, an initial anomaly detection model can be generated.

[0053] In the above implementation, a model based on a spatiotemporal hypergraph neural network is used to encode existing time series and predict future system states, which can effectively characterize and utilize the time dependency and spatial dependency of data, help to better fit the training data, provide more accurate prediction values, and provide guarantees for subsequent anomaly detection.

[0054] S105: Using the target data set and the mean square error loss function, the initial anomaly detection model is trained and verified to generate a target anomaly detection model.

[0055] It should be noted that the target data set refers to the multivariate time series collected by different sensors when the target CPS system is operating normally, that is, there is no abnormal data in the training and verification data. Through the unsupervised learning mechanism, using normal data To perform offline training and verification on the anomaly detection model constructed in the above steps.

[0056] In some specific implementations, training and validating the initial anomaly detection model using a target data set and a mean square error loss function to generate a target anomaly detection model includes: The target data set is divided into a training set and a validation set, where the division ratio can be set according to actual needs, such as 80% for the training set and 20% for the validation set, and the target data set is divided into windows of length Window data , to predict the state of the last step of the window data , where the window length can be set according to actual needs, and the preferred value is ; Based on the adaptive optimizer, the training set and the validation set, the initial anomaly detection model is trained and validated, and based on the mean square error loss function, the loss value corresponding to the validation set is determined, wherein the training method adopts the gradient descent method, and the adaptive optimizer such as Adam is used to optimize it, and the mean square error loss function includes:

[0057] in, Represents the time series length of all training data; According to the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches the preset standard, and the target anomaly detection model is output, wherein the initial learning rate, the learning rate adjustment value and the preset standard can be set according to actual needs. Exemplarily, the learning rate is set to 0.001, and the learning rate is adjusted according to the loss value of the validation set, with each adjustment multiple being 0.5 until the learning rate reaches 0.00001. This application uses an early stopping mechanism to avoid model overfitting, and the patience value of the early stopping mechanism is set to 10.

[0058] In the above implementation, the constructed anomaly detection model is trained without relying on any labeled anomaly data, which effectively avoids the problem of difficulty in obtaining real anomaly data and high labeling cost, reduces the implementation difficulty, and improves the scope of application of the anomaly detection model.

[0059] In the above-mentioned multivariate time series anomaly detection method, the method includes: initializing the learnable embedding vector of the device node, and determining the hyperedge of the first hypergraph structure based on the similarity between the learnable embedding vectors, wherein the hypergraph structure is composed of device nodes and hyperedges; based on the approximate discrete optimization mechanism, adjusting the device nodes and hyperedges in the first hypergraph structure respectively to obtain a second hypergraph structure; modeling the learnable embedding vector, the second hypergraph structure and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is obtained through sensors on the device nodes; combining the spatiotemporal hypergraph neural network model and the fully connected neural network to generate an initial anomaly detection model The fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model; the initial anomaly detection model is trained and verified using the target data set and the mean square error loss function to generate a target anomaly detection model. This application can better characterize the complex high-order interaction relationship between different device nodes in the real system by introducing the hypergraph structure into the anomaly detection problem of multivariate time series, which is conducive to the evolution of the fitting system. The multivariate time series is modeled by the spatiotemporal hypergraph neural network, and the spatial relationship and timing information are modeled, so as to better complete the multivariate time series anomaly detection task and improve the detection efficiency and accuracy.

[0060] It should be understood that although Figure 3-Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3-Figure 4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0061] In one embodiment, Figure 5 As shown in the figure, a multivariate time series anomaly detection method is provided. Figure 2 The terminal in is used as an example to illustrate, including the following steps: S201, obtaining a multivariate time series of device nodes in a system to be detected.

[0062] It should be noted that the system to be detected refers to the CPS system, but it can also be other information-physical systems that require anomaly detection. The multivariate time series is obtained through sensors on the device nodes.

[0063] S202: Input the multivariate time series into the target anomaly detection model constructed by the above method, and output a predicted value.

[0064] It should be noted that, generally after the target anomaly detection model is generated, it can be deployed to the corresponding system to predict the state of the system at the next moment based on the collected multivariate time series. The deployed system may include information-physical systems such as water resource detection systems, power grid detection systems, and server detection systems. Among them, the target anomaly detection model can predict the "normal" system state. When the actual system state deviates greatly from the predicted system state, it can be considered that the actual system state is in an "abnormal" state. Based on this, the system's abnormal conditions can be detected.

[0065] S203: Calculate the deviation value between the predicted value and the true value, and determine the state of the system to be detected based on a comparison result between a first target deviation value and a first preset threshold.

[0066] It should be noted that the predicted value refers to the output value of the target anomaly detection model, the true value refers to the system state value obtained at the current time node, and the value of the first preset threshold is the maximum error value corresponding to all moments in the process of verifying the anomaly detection model based on the verification set.

[0067] In some specific embodiments, after calculating the deviation value between the predicted value and the true value, the method further includes: Performing robust normalization processing on the deviation value to obtain a first target node deviation value; The multiple first target node deviation values ​​of the system at the same time node are sorted, and the first target deviation value is determined according to the sorting result, that is, the maximum value in the sorting result is selected and defined as the first target deviation value.

[0068] Specifically, through Representation Node At the moment The deviation between the predicted and true values ​​is the value of the error. Since different sensors (nodes) may have very different characteristics, their deviation values ​​may also have very different scales. In order to prevent the deviation generated by any one sensor from excessively dominating other sensors, the error value of each sensor is robustly normalized. The formula is:

[0069] in, and Respectively The median and quartiles of the values ​​across time scales. This application uses the median and quartiles instead of the mean and standard deviation to improve the robustness of anomaly detection; Further, using The maximum value of all node error values ​​at a given moment is used as the basis for judging whether an error occurs in the system at that moment, that is, .

[0070] In some specific implementations, determining the state of the system to be detected based on a comparison result between the first target deviation value and the first preset threshold value includes: Obtaining a second target node deviation value obtained in the process of verifying the initial anomaly detection model, wherein the second target node deviation value refers to the error value corresponding to all moments in the process of verifying the anomaly detection model based on the verification set; Sort the deviation values ​​of the second target nodes corresponding to multiple time nodes in the target time window, and determine the first preset threshold value according to the sorting result, that is, the maximum error value reached at all times on the verification set. Defined as a first preset threshold; In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state, that is, and For comparison, Greater than threshold , it means that at the predicted time The system fails, that is, the system is in an abnormal state.

[0071] In the above implementation, the system state is predicted by constructing a target anomaly detection model, and the final state of the system is evaluated by combining the prediction results, the Robustin normalization mechanism and the threshold determined based on the validation set, thereby further improving the accuracy of multivariate time series anomaly detection.

[0072] In some embodiments, the method further comprises: Obtain the robustly normalized error values ​​corresponding to the sensors at multiple time nodes, and calculate the first perception coefficients corresponding to the multiple error values. The number of time nodes can be determined according to actual needs, and the error value calculation formula is:

[0073] in, represents the first perceptual coefficient, Indicates the number of error values, , Respectively represent and Error value; In response to detecting a first perceptual coefficient When the value is less than the second preset threshold, a second perception coefficient corresponding to the sensor is determined, wherein the second preset threshold can be set according to actual needs, and the calculation method of the second perception coefficient includes:

[0074] in, represents the second perceptual coefficient, Indicates the deviation value; In response to detecting that the second perception coefficient is less than a third preset threshold, the device node corresponding to the sensor is marked, and in subsequent multivariate time series anomaly detection, the device node corresponding to the sensor is directly marked. The calculated deviation value is used in the sorting process to determine whether an error occurs in the system at that moment, wherein the third preset threshold value can be determined according to actual needs.

[0075] In the above implementation, by marking the device node corresponding to the sensor, there is no need to perform robust normalization on the error value of the sensor in the subsequent abnormality detection process, thereby improving the detection efficiency while ensuring the detection accuracy.

[0076] It should be understood that although Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0077] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multivariate time series anomaly detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0078] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0079] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: S101, initializing learnable embedding vectors of device nodes, and determining hyperedges of a first hypergraph structure based on similarities between learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; S102, based on an approximate discrete optimization mechanism, adjusting the device nodes and hyperedges in the first hypergraph structure respectively to obtain a second hypergraph structure; S103, modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes; S104, generating an initial anomaly detection model by combining the spatiotemporal hypergraph neural network model and the fully connected neural network, wherein the fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model; S105: Using the target data set and the mean square error loss function, the initial anomaly detection model is trained and verified to generate a target anomaly detection model.

[0080] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Using the vector similarity function, calculate the similarity between the target learnable embedding vector and other learnable embedding vectors; Sorting the similarities in descending order to obtain a sorting result, and selecting a target number of device nodes based on the sorting result; Taking the device node corresponding to the target learnable embedding vector as the center, constructing a hyperedge including the target number of device nodes; Based on the device nodes and the hyperedges, the first hypergraph structure is generated.

[0081] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Initialize the learnable sampling parameters of the hyperedge; Based on a discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0082] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Define the hidden state of the device node and the hidden state of the hyperedge; Calculating the similarity between the hidden state of the device node and the hidden state of the hyperedge; Determining an increment matrix and a mask matrix of the hyperedge according to the similarity; Based on the hyperedge mask, the hyperedge increment matrix and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain a second hypergraph structure.

[0083] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Determining a device node feature matrix based on the learnable embedding vectors of the device nodes and the multivariate time series; Determine a hypergraph neural network for representing hypergraph data, wherein the hypergraph data is a time series consisting of the device node feature matrix and the second hypergraph structure; determining an attention mechanism model for processing the time series; The hypergraph neural network and the attention mechanism model are combined to generate the spatiotemporal hypergraph neural network model.

[0084] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device nodes; Repeat the iterative aggregation operation to obtain the hidden states of nodes and hyperedges at the target time within the target time window.

[0085] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: The hidden states of nodes and hyperedges at multiple target moments in the target time window are aggregated to obtain a time series of nodes and hyperedges that fuse the spatial relationships of the second hypergraph structure in the target time window.

[0086] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0087] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Dividing the target data set into a training set and a validation set; Based on the adaptive optimizer, the training set and the validation set, the initial anomaly detection model is trained and validated, and based on the mean square error loss function, a loss value corresponding to the validation set is determined; According to the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and the target anomaly detection model is output.

[0088] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: S201, obtaining a multivariate time series of device nodes in the system to be detected; S202, inputting the multivariate time series into the target anomaly detection model constructed by the above method, and outputting a predicted value; S203: Calculate the deviation value between the predicted value and the true value, and determine the state of the system to be detected based on a comparison result between a first target deviation value and a first preset threshold.

[0089] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Performing robust normalization processing on the deviation value to obtain a first target node deviation value; Sorting multiple first target node deviation values ​​of the system at the same time node, and determining the first target deviation value according to the sorting result.

[0090] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: Obtaining a second target node deviation value obtained in the process of verifying the initial anomaly detection model; Sorting the second target node deviation values ​​corresponding to multiple time nodes in the target time window, and determining a first preset threshold value according to the sorting result; In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0091] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: S101, initializing learnable embedding vectors of device nodes, and determining hyperedges of a first hypergraph structure based on similarities between learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; S102, based on an approximate discrete optimization mechanism, adjusting the device nodes and hyperedges in the first hypergraph structure respectively to obtain a second hypergraph structure; S103, modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes; S104, generating an initial anomaly detection model by combining the spatiotemporal hypergraph neural network model and the fully connected neural network, wherein the fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model; S105: Using the target data set and the mean square error loss function, the initial anomaly detection model is trained and verified to generate a target anomaly detection model.

[0092] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Using the vector similarity function, calculate the similarity between the target learnable embedding vector and other learnable embedding vectors; Sorting the similarities in descending order to obtain a sorting result, and selecting a target number of device nodes based on the sorting result; Taking the device node corresponding to the target learnable embedding vector as the center, constructing a hyperedge including the target number of device nodes; Based on the device nodes and the hyperedges, the first hypergraph structure is generated.

[0093] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Initialize the learnable sampling parameters of the hyperedge; Based on a discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0094] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Define the hidden state of the device node and the hidden state of the hyperedge; Calculating the similarity between the hidden state of the device node and the hidden state of the hyperedge; Determining an increment matrix and a mask matrix of the hyperedge according to the similarity; Based on the hyperedge mask, the hyperedge increment matrix and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain a second hypergraph structure.

[0095] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Determining a device node feature matrix based on the learnable embedding vectors of the device nodes and the multivariate time series; Determine a hypergraph neural network for representing hypergraph data, wherein the hypergraph data is a time series consisting of the device node feature matrix and the second hypergraph structure; determining an attention mechanism model for processing the time series; The hypergraph neural network and the attention mechanism model are combined to generate the spatiotemporal hypergraph neural network model.

[0096] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device nodes; Repeat the iterative aggregation operation to obtain the hidden states of nodes and hyperedges at the target time within the target time window.

[0097] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: The hidden states of nodes and hyperedges at multiple target moments in the target time window are aggregated to obtain a time series of nodes and hyperedges that fuse the spatial relationships of the second hypergraph structure in the target time window.

[0098] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0099] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Dividing the target data set into a training set and a validation set; Based on the adaptive optimizer, the training set and the validation set, the initial anomaly detection model is trained and validated, and based on the mean square error loss function, a loss value corresponding to the validation set is determined; According to the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and the target anomaly detection model is output.

[0100] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: S201, obtaining a multivariate time series of device nodes in the system to be detected; S202, inputting the multivariate time series into the target anomaly detection model constructed by the above method, and outputting a predicted value; S203: Calculate the deviation value between the predicted value and the true value, and determine the state of the system to be detected based on a comparison result between a first target deviation value and a first preset threshold.

[0101] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Performing robust normalization processing on the deviation value to obtain a first target node deviation value; Sorting multiple first target node deviation values ​​of the system at the same time node, and determining the first target deviation value according to the sorting result.

[0102] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Obtaining a second target node deviation value obtained in the process of verifying the initial anomaly detection model; Sorting the second target node deviation values ​​corresponding to multiple time nodes in the target time window, and determining a first preset threshold value according to the sorting result; In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0103] In one embodiment, a computer program product is provided, the computer program product comprising a computer program, the computer program when executed by a processor implements the following steps: S101, initializing learnable embedding vectors of device nodes, and determining hyperedges of a first hypergraph structure based on similarities between learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; S102, based on an approximate discrete optimization mechanism, adjusting the device nodes and hyperedges in the first hypergraph structure respectively to obtain a second hypergraph structure; S103, modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes; S104, generating an initial anomaly detection model by combining the spatiotemporal hypergraph neural network model and the fully connected neural network, wherein the fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model; S105: Using the target data set and the mean square error loss function, the initial anomaly detection model is trained and verified to generate a target anomaly detection model.

[0104] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Using the vector similarity function, calculate the similarity between the target learnable embedding vector and other learnable embedding vectors; Sorting the similarities in descending order to obtain a sorting result, and selecting a target number of device nodes based on the sorting result; Taking the device node corresponding to the target learnable embedding vector as the center, constructing a hyperedge including the target number of device nodes; Based on the device nodes and the hyperedges, the first hypergraph structure is generated.

[0105] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Initialize the learnable sampling parameters of the hyperedge; Based on a discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

[0106] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Define the hidden state of the device node and the hidden state of the hyperedge; Calculating the similarity between the hidden state of the device node and the hidden state of the hyperedge; Determining an increment matrix and a mask matrix of the hyperedge according to the similarity; Based on the hyperedge mask, the hyperedge increment matrix and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain a second hypergraph structure.

[0107] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Determining a device node feature matrix based on the learnable embedding vectors of the device nodes and the multivariate time series; Determine a hypergraph neural network for representing hypergraph data, wherein the hypergraph data is a time series consisting of the device node feature matrix and the second hypergraph structure; determining an attention mechanism model for processing the time series; The hypergraph neural network and the attention mechanism model are combined to generate the spatiotemporal hypergraph neural network model.

[0108] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device nodes; Repeat the iterative aggregation operation to obtain the hidden states of nodes and hyperedges at the target time within the target time window.

[0109] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: The hidden states of nodes and hyperedges at multiple target moments in the target time window are aggregated to obtain a time series of nodes and hyperedges that fuse the spatial relationships of the second hypergraph structure in the target time window.

[0110] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

[0111] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Dividing the target data set into a training set and a validation set; Based on the adaptive optimizer, the training set and the validation set, the initial anomaly detection model is trained and validated, and based on the mean square error loss function, a loss value corresponding to the validation set is determined; According to the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and the target anomaly detection model is output.

[0112] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: S201, obtaining a multivariate time series of device nodes in the system to be detected; S202, inputting the multivariate time series into the target anomaly detection model constructed by the above method, and outputting a predicted value; S203: Calculate the deviation value between the predicted value and the true value, and determine the state of the system to be detected based on a comparison result between a first target deviation value and a first preset threshold.

[0113] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Performing robust normalization processing on the deviation value to obtain a first target node deviation value; Sorting multiple first target node deviation values ​​of the system at the same time node, and determining the first target deviation value according to the sorting result.

[0114] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: Obtaining a second target node deviation value obtained in the process of verifying the initial anomaly detection model; Sorting the second target node deviation values ​​corresponding to multiple time nodes in the target time window, and determining a first preset threshold value according to the sorting result; In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A model building method, characterized in that: The method comprises: Initializing learnable embedding vectors of device nodes, and determining hyperedges of a first hypergraph structure based on similarities between learnable embedding vectors, wherein the hypergraph structure consists of device nodes and hyperedges; Based on the approximate discrete optimization mechanism, the device nodes and the hyperedges in the first hypergraph structure are adjusted respectively to obtain a second hypergraph structure; Modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model, wherein the multivariate time series is acquired through sensors on the device nodes; Combining the spatiotemporal hypergraph neural network model and the fully connected neural network to generate an initial anomaly detection model, wherein the fully connected neural network is used to predict the state of the device node at the next moment according to the output value of the spatiotemporal hypergraph neural network model; The initial anomaly detection model is trained and verified using the target data set and the mean square error loss function to generate a target anomaly detection model.

2. The model building method according to claim 1, characterized in that: Based on the similarity between the learnable embedding vectors, determining the hyperedges of the first hypergraph structure includes: Using the vector similarity function, calculate the similarity between the target learnable embedding vector and other learnable embedding vectors; Sorting the similarities in descending order to obtain a sorting result, and selecting a target number of device nodes based on the sorting result; Taking the device node corresponding to the target learnable embedding vector as the center, constructing a hyperedge including the target number of device nodes; Based on the device nodes and the hyperedges, the first hypergraph structure is generated.

3. The model building method according to claim 1, characterized in that: Based on the approximate discrete optimization mechanism, the device nodes and hyperedges in the first hypergraph structure are adjusted respectively, and the second hypergraph structure is obtained, including: Initialize the learnable sampling parameters of the hyperedge; Based on a discrete distribution sampling mechanism, the learnable sampling parameters of the hyperedge are processed to generate a hyperedge mask.

4. The model building method according to claim 3, characterized in that: Based on the approximate discrete optimization mechanism, the device nodes and the hyperedges in the first hypergraph structure are adjusted respectively, and the second hypergraph structure obtained further includes: Define the hidden state of the device node and the hidden state of the hyperedge; Calculating the similarity between the hidden state of the device node and the hidden state of the hyperedge; Determining an increment matrix and a mask matrix of the hyperedge according to the similarity; Based on the hyperedge mask, the hyperedge increment matrix and the mask matrix, the device nodes and hyperedges in the first hypergraph structure are adjusted to obtain a second hypergraph structure.

5. The model building method according to claim 1, characterized in that: Modeling the learnable embedding vector, the second hypergraph structure, and the multivariate time series to generate a spatiotemporal hypergraph neural network model includes: Determining a device node feature matrix based on the learnable embedding vectors of the device nodes and the multivariate time series; Determine a hypergraph neural network for representing hypergraph data, wherein the hypergraph data is a time series consisting of the device node feature matrix and the second hypergraph structure; determining an attention mechanism model for processing the time series; The hypergraph neural network and the attention mechanism model are combined to generate the spatiotemporal hypergraph neural network model.

6. The model building method according to claim 5, characterized in that: Methods for representing hypergraph data based on hypergraph neural networks include: Aggregate the learnable embedding vectors of device nodes to the corresponding hyperedges; Aggregate the features of the hyperedge to the corresponding device nodes; Repeat the iterative aggregation operation to obtain the hidden states of nodes and hyperedges at the target time within the target time window.

7. The model building method according to claim 6, characterized in that: The method further comprises: The hidden states of nodes and hyperedges at multiple target moments in the target time window are aggregated to obtain a time series of nodes and hyperedges that fuse the spatial relationships of the second hypergraph structure in the target time window.

8. The model building method according to claim 5, characterized in that: Methods for processing time series based on attention mechanism models include: Based on the attention mechanism, the time series is encoded to obtain the hidden states of the target node and the target hyperedge.

9. The model building method according to claim 1, characterized in that: Using the target data set and the mean square error loss function, the initial anomaly detection model is trained and verified to generate a target anomaly detection model including: Dividing the target data set into a training set and a validation set; Based on the adaptive optimizer, the training set and the validation set, the initial anomaly detection model is trained and validated, and based on the mean square error loss function, a loss value corresponding to the validation set is determined; According to the loss value, the learning rate of the adaptive optimizer is adjusted until the learning rate reaches a preset standard, and the target anomaly detection model is output.

10. A multivariate time series anomaly detection method, characterized in that: The method comprises: Obtain multivariate time series of device nodes in the system to be detected; Inputting the multivariate time series into a target anomaly detection model constructed by the method of any one of claims 1 to 9, and outputting a predicted value; The deviation value between the predicted value and the true value is calculated, and based on the comparison result between the first target deviation value and the first preset threshold value, the state of the system to be detected is determined.

11. The multivariate time series anomaly detection method according to claim 10, characterized in that: After calculating the deviation value between the predicted value and the true value, the method further includes: Performing robust normalization processing on the deviation value to obtain a first target node deviation value; Sorting multiple first target node deviation values ​​of the system at the same time node, and determining the first target deviation value according to the sorting result.

12. The multivariate time series anomaly detection method according to claim 10, characterized in that: Determining the state of the system to be detected based on a comparison result between the first target deviation value and the first preset threshold value includes: Obtaining a second target node deviation value obtained in the process of verifying the initial anomaly detection model; Sorting the second target node deviation values ​​corresponding to multiple time nodes in the target time window, and determining a first preset threshold value according to the sorting result; In response to detecting that the first target deviation value is greater than the first preset threshold, it is determined that the system to be detected is in an abnormal state.

13. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 12 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images