Wind turbine multi-dimensional time series data anomaly detection method and system

The spatial and temporal characteristics of wind turbine multi-dimensional timing data are extracted through graph attention network and Transformer, and feature fusion and reconstruction are combined with gated cycle units and bidirectional long and short-term networks, which solves the construction of time and space correlation in the existing technology The problem of difficulty in modeling significantly improves the accuracy of abnormal detection.

CN118656756BActive Publication Date: 2024-12-03HUNAN UNIV OF SCI & TECH
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411116870.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

The prior art is difficult to simultaneously effectively model and feature fusion of the time correlation and spatial correlation of multi-dimensional timing data of wind turbines, resulting in low accuracy of abnormal detection.

Method used

The graph attention network and Transformer are used to extract the spatial and temporal dependency characteristics of time series data respectively, and feature fusion and reconstruction are performed through gated loop units and bidirectional long and short-term networks to calculate the reconstruction error to judge abnormalities.

Benefits of technology

The accuracy of the abnormal detection of multi-dimensional timing data of wind turbines is improved, and the time and space correlation characteristics are captured in parallel and detailed supplements are supplemented, and the characteristics are fused to enhance detection capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118656756B_ABST
    Figure CN118656756B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for detecting abnormalities in multi-dimensional time series data of wind turbines, which obtains the time series of multi-dimensional operation data of wind turbines; uses a graph attention network to extract the spatial dependency features of the time series; uses a transformer to extract the time dependency features of the time series; uses a gated recurrent unit to fuse the spatial dependency features with the time dependency features to obtain spatiotemporal dependency features; uses a bidirectional long-short term network to reconstruct the spatiotemporal dependency features, and calculates the reconstruction error of the spatiotemporal dependency features, and determines whether the time series has abnormalities according to the reconstruction error. The present invention can greatly improve the accuracy of abnormality detection in multi-dimensional time series data of wind turbines.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of data processing and data transmission, and in particular to a method and system for detecting anomalies in multi-dimensional time series data of a wind turbine. Background Art

[0002] Wind turbine SCADA systems record a large amount of multidimensional time series data generated by sensors. The changes in these sensors reflect the system's operating status, and the values ​​of multiple sensors are interrelated. This means that multidimensional time series anomaly detection requires not only considering the transformation of time series in each dimension across time scales, but also the correlation between the transformations of time series in different dimensions within the same time period. The key to detecting anomalies in multidimensional time series is achieving efficient and accurate computation. With the development of deep learning technology, data-driven unsupervised deep learning has been widely used for multidimensional time series anomaly detection.

[0003] However, due to the complexity of multidimensional time series, existing methods struggle to simultaneously model both temporal and spatial correlations in wind power monitoring data. In recent years, transformers and graph neural networks have been widely used in multidimensional time series anomaly detection. These new deep learning models can model both temporal and spatial correlations to a certain extent, but they still struggle to combine both and fuse their features.

[0004] Therefore, how to achieve the fusion of temporal correlation and spatial correlation characteristics of multidimensional time series and thus improve the accuracy of anomaly detection in multidimensional time series data of wind turbines has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] The present invention provides a method and system for detecting anomalies in multi-dimensional time series data of a wind turbine, which are used to solve the technical problem of low accuracy in detecting anomalies in multi-dimensional time series data of a wind turbine.

[0006] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0007] A method for detecting anomalies in multi-dimensional time series data of a wind turbine, comprising:

[0008] Obtaining time series of multi-dimensional operation data of wind turbines;

[0009] A graph attention network is used to extract the spatial dependency features of the time series; a transformer is used to extract the temporal dependency features of the time series;

[0010] A gated recurrent unit is used to fuse the spatial dependency feature and the temporal dependency feature to obtain a spatiotemporal dependency feature;

[0011] A bidirectional long-short-term network is used to reconstruct the spatiotemporal dependency features, and a reconstruction error of the spatiotemporal dependency features is calculated. It is then determined whether the time series has an anomaly based on the reconstruction error.

[0012] Preferably, a gated recurrent unit is used to fuse the spatial dependency feature and the temporal dependency feature to obtain a spatiotemporal dependency feature, including:

[0013] Using skip connections, the temporal sequence, spatial dependency features, and time dependency features are directly concatenated to obtain concatenated features;

[0014] The concatenated features are fused using a gated recurrent unit, and spatiotemporal dependency features are extracted from the fused features.

[0015] Preferably, the skip connection is expressed as:

[0016] ;

[0017] in, represents the concatenation of two vectors. Represents spatial dependence characteristics, is a time series; is a time-dependent feature;

[0018] and / or

[0019] The splicing features are fused using a gated recurrent unit, and spatiotemporal dependency features are extracted from the fused features, which is achieved by the following formula:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] in, Represents the update gate, which controls the amount of data that can be retained from the previous moment to the current moment; Represents the activation function relu; Represents the update gate weight matrix; Indicates the output at the last moment; Represents data at the current moment; Reset gate. Represents resetting the gate weight matrix; The weight matrix representing the state to be selected; Indicates the output at the current moment; Indicates the pending status.

[0025] Preferably, spatially dependent features Satisfied with the following formula:

[0026] ,

[0027] ;

[0028] in, Represents the activation function relu; Represents the number of multi-head attention mechanisms in the graph attention network; Indicates the sequence number of the multi-head attention mechanism; , Represent two nodes in the graph structure respectively; Representation and Node A collection of connected nodes; Representation and Node connected nodes; Indicates the Node under the head attention mechanism j With node The attention coefficient; Indicates the h Linear transformation matrix under the head attention mechanism; Representation and Node Connected nodes The eigenvector of represents the activation function tanh; represents a learnable weight vector; represents the transpose of the learnable weight vector; represents the linear transformation matrix; Representation node The eigenvector of Representation node j The feature vector of || represents the information of two nodes respectively.

[0029] Preferably, the reconstruction error is achieved by the following steps:

[0030] Calculate the mean square error between the original data and the reconstructed data;

[0031] Calculate the cosine distance value of the covariance between the original data and the reconstructed data;

[0032] The mean square error value and the cosine distance value are fused to obtain the reconstruction error.

[0033] Preferably, the reconstruction error satisfies:

[0034] ;

[0035] in, is the reconstruction error, is the mean square error between the time series and the reconstructed data, is the cosine similarity value; For control Hyperparameters of weights.

[0036] Preferably, the Satisfied with:

[0037] ;

[0038] in, is the number of time points in the time series, is the number of variables in the time series, a represents the time point number, b represents the variable number, is the sample matrix of the original data, is the sample matrix of the reconstructed data;

[0039] and / or

[0040] described Satisfied with:

[0041] ;

[0042] ;

[0043] in, Indicates the degree of similarity between two vectors in direction; represents cosine similarity; Represents the covariance of the original time series data; represents the covariance of the reconstructed data; represents the Euclidean norm; is the default value.

[0044] Preferably, the bidirectional long-short term network adopts an unsupervised learning strategy, uses reconstruction error as anomaly score, and uses a point adjustment strategy to adjust anomaly labels.

[0045] A computer system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0046] The present invention has the following beneficial effects:

[0047] This method uses a transformer and a graph neural network to capture the temporal and spatial correlation features of multidimensional time series data in parallel. It uses skip connections to supplement detailed information and gated recurrent units to fuse temporal and spatial correlation features. Finally, it uses a bidirectional long short-term memory network to reconstruct the data. The loss function uses the sum of the mean square error loss and the cosine similarity of the covariance between the reconstructed data and the original data as the model's loss function. This significantly improves the accuracy of anomaly detection in multidimensional time series data from wind turbines.

[0048] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0050] Figure 1 A flow chart showing the steps of a method for detecting abnormalities in a multi-dimensional time series of a wind turbine according to the present invention is provided;

[0051] Figure 2 A training model flow chart for constructing a method for detecting anomalies in a multi-dimensional time series of a wind turbine according to the present invention is shown;

[0052] Figure 3 The present invention shows a specific flow chart of abnormality detection of multi-dimensional time series of wind turbines;

[0053] Figure 4 A block diagram of a wind turbine multi-dimensional time series anomaly detection system according to the present invention is shown. DETAILED DESCRIPTION

[0054] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered by the claims.

[0055] like Figure 1 、 3 As shown, the present invention discloses a method for detecting abnormalities in a multi-dimensional time series of a wind turbine, comprising:

[0056] S102, obtaining a multi-dimensional time series of a wind turbine;

[0057] S104 uses a parallel graph attention network and transformer to capture spatial and temporal dependencies respectively;

[0058] S106, using a gated recurrent unit to further extract and fuse spatiotemporal dependency features;

[0059] S108, reconstructing data using a bidirectional long-term and short-term network;

[0060] S110 , calculating a reconstruction error as an anomaly score.

[0061] According to the embodiment of the present invention, the model parameters are determined based on the characteristics of the correlation of multi-dimensional time series data, a training model is established, and the long-term stable operation equipment monitoring data is used to The model is trained continuously. The purpose of the model is to extract the time correlation and spatial correlation in the time series data based on the input data, extract the fusion features of the time and space correlation, and then reconstruct the features into a time series sequence. .use and The sum of the mean square error loss and its covariance cosine distance is used as the output value of the model, and the anomaly score.

[0062] According to an embodiment of the present invention, Figure 2 As shown above, the parallel graph attention network and transformer are used to capture spatial dependencies and temporal dependencies respectively, including:

[0063] S202, using a graph attention network to capture the spatial dependency features between variables in the multi-dimensional time series data of wind turbines;

[0064] S204: Use transformer to capture the time dependency characteristics between variables in the multi-dimensional time series data of wind turbines.

[0065] It should be noted that in this method, the samples are mapped into graph structure data, and each subsequence is regarded as a node. d Dimensional Moment t The subsequence of Weighted undirected graph ,in represents a node set, Represents the edge set. Any two nodes i and j have a connection relationship, and the adjacency matrix is ​​recorded as ,in , then the attention coefficient ,in Represents the activation function LeakyReLU, Represents the learnable weight vector, W represents the shared weight matrix, || represents the information of two nodes before and after, and finally aggregates the adjacent nodes to obtain the output of each node , where i represents the node number and H represents the number of multi-head attention mechanisms. To facilitate training, the averaging method is used to summarize the results of the multi-head attention mechanism. At the same time, Transformer is used to capture the temporal correlation characteristics of multi-dimensional time series data.

[0066] According to an embodiment of the present invention, further extracting and fusing spatiotemporal dependency features using a gated recurrent unit includes:

[0067] Use skip connections to directly concatenate the original input with the spatiotemporal dependency features;

[0068] The splicing features are further extracted and fused with spatiotemporal dependency features using a gated recurrent unit.

[0069] It should be noted that the original data is recorded as , the temporal correlation features captured by Transformer are recorded as , the spatial correlation captured by the graph attention network is recorded as , then the skip connection can be expressed as ,in Indicates the connection of two vectors. Use gated recurrent units for feature extraction and fusion. Represents the data at the current moment, Indicates the output at the last moment, Indicates the current moment output, Indicates the state of waiting for selection; Represents matrix addition. Part represents the reset gate, which controls how much past information to forget. The part represents the update gate, which controls the amount of data that can be retained from the previous moment to the current moment. Its calculation formula can be expressed as , ,in represents the update gate weight matrix, Represents resetting the gate weight matrix, The weight matrix representing the state to be selected.

[0070] According to an embodiment of the present invention, using a bidirectional long-term and short-term network to reconstruct data includes:

[0071] Use a bidirectional long-short-term network to reconstruct the fused features into a time series;

[0072] It should be noted that due to the difficulty of data labeling, this method uses an unsupervised learning approach. During the training phase, data with long-term stable operation is treated as normal sample data. During the training phase, the model learns the correlation characteristics of the normal sample multidimensional time series data and reconstructs it into multidimensional time series data. Then, for the test data, the reconstruction error is used as the anomaly score, and the anomaly labels are adjusted using a point adjustment strategy.

[0073] According to an embodiment of the present invention, calculating the reconstruction error as a loss function and an anomaly score includes:

[0074] Calculate the mean square error between the original data and the reconstructed data;

[0075] Calculate the cosine distance of the covariance between the original data and the reconstructed data;

[0076] Combine the cosine distance of covariance and mean square error as the loss function of the model and the anomaly score;

[0077] It should be noted that in statistics and probability theory, the covariance matrix is ​​a square matrix whose elements at positions i and j are the covariances between the i-th and j-th random variables, reflecting the degree of correlation between variables of different dimensions in a sample. This is a natural extension from scalar random variables to high-dimensional random vectors. We use Representing the covariance of a sample, we record n ​​time points and k-dimensional features as , then the mean of all features can be recorded as , after subtracting the mean from the sample, we can get , then the covariance calculation formula can be expressed as , cosine similarity measures the similarity between two vectors by measuring the cosine value of the angle between them. The cosine value of a 0-degree angle is 1, while the cosine value of any other angle is not greater than 1; and its minimum value is -1. Therefore, the cosine value of the angle between two vectors determines whether the two vectors point in roughly the same direction. When two vectors have the same direction, the cosine similarity value is 1; when the angle between the two vectors is 90°, the cosine similarity value is 0; when the two vectors point in completely opposite directions, the cosine similarity value is -1. The formula for calculating cosine similarity is , where A and B represent two different vectors, Indicates the degree of similarity between two vectors in terms of direction. Cosine similarity is independent of the length of the vector and is only related to the direction of the vector. The loss function of the training model. The mean square error is used to measure the sample matrix. and reconstructed matrix The numerical similarity between them is calculated as follows:

[0078] ;

[0079] in, is the number of time points in the time series, is the number of variables in the time series, a represents the time point number, b represents the variable number, is the sample matrix of the original data, is the sample matrix of the reconstructed data

[0080] Next, we calculate the sample matrix and reconstructed matrix The covariance matrix of and , and calculate the cosine similarity of its covariance, the calculation formula is Since the range of mean square error and cosine similarity are different and their meanings are also different, we transform cosine similarity as follows: , the transformed cosine distance and mean square error have the same value range as , and have the same meaning: 0 means the most similar and the larger the value, the lower the similarity. ,in, For control The hyperparameters of weights, It is a preset value, preferably 0.5.

[0081] like Figure 4 As shown, this embodiment also provides a computer system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0082] In summary, the present invention uses a parallel graph attention network and transformer to capture the spatial and temporal dependencies between variables in the multidimensional time series generated by wind turbines. Skip connections are then used to directly concatenate the original input with the spatiotemporal dependencies, followed by a gated recurrent unit to further extract and fuse the spatiotemporal dependency features. Finally, a bidirectional long-short-term network is used to reconstruct the data. To make the reconstructed data more similar to the original data distribution, the cosine distance of the covariance between the original and reconstructed data is combined with the mean square error loss as the model's loss function and reconstruction error calculation method. The present invention was experimented on real-world datasets and achieved good experimental results.

[0083] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for detecting anomalies in multi-dimensional time series data of a wind turbine, characterized in that: include: Obtaining the time series of multi-dimensional operation data of wind turbines; Using a graph attention network to extract spatial dependency features of the temporal sequence; Using transformer to extract time-dependent features of the time series; A gated recurrent unit is used to fuse the spatial dependency feature and the temporal dependency feature to obtain a spatiotemporal dependency feature; Reconstructing the spatiotemporal dependency features using a bidirectional long-short term network, calculating a reconstruction error of the spatiotemporal dependency features, and judging whether the time series has an abnormality according to the reconstruction error; The space-dependent features and the time-dependent features are fused by using a gated recurrent unit to obtain spatiotemporal-dependent features, including: Using skip connections, the temporal sequence, the spatial dependent features, and the time dependent features are directly concatenated to obtain concatenated features; The concatenated features are fused using a gated recurrent unit, and spatiotemporal dependency features are extracted from the fused features.

2. The method for detecting abnormality of multi-dimensional time series data of a wind turbine according to claim 1, characterized in that: The skip connection is expressed as: ; in, represents the concatenation of two vectors. Represents spatial dependence characteristics, is a time series; is a time-dependent feature; and / or The concatenated features are fused using a gated recurrent unit, and spatiotemporal dependency features are extracted from the fused features, which is achieved by the following formula: ; ; ; ; in, Represents the update gate, which controls the amount of data that can be retained from the previous moment to the current moment; Represents the activation function relu; Represents the update gate weight matrix; Indicates the output at the last moment; Represents data at the current moment; Reset gate. Represents resetting the gate weight matrix; The weight matrix representing the state to be selected; Indicates the current moment output; Indicates the pending status.

3. The method for detecting abnormality of multi-dimensional time series data of a wind turbine according to claim 2, characterized in that: Spatial Dependence Features Satisfies the following formula: , ; in, Represents the activation function relu; Represents the number of multi-head attention mechanisms in the graph attention network; Indicates the serial number of the multi-head attention mechanism; , Respectively represent two nodes in the graph structure; Representation and Node A collection of connected nodes; Representation and Node connected nodes; Representation Node j With Node The attention coefficient; Indicates Node under the head attention mechanism j With Node The attention coefficient; Indicates h Linear transformation matrix under the head attention mechanism; Representation and Node Connected Nodes The eigenvector of represents the activation function tanh; represents a learnable weight vector; represents the transpose of the learnable weight vector; represents a linear transformation matrix; Representation Node The eigenvector of Representation Node j The feature vector of ; the || before and after represent the information of two nodes respectively.

4. The method for detecting abnormality of multi-dimensional time series data of a wind turbine according to claim 1, characterized in that: The reconstruction error is achieved by the following steps: Calculate the mean square error between the original data and the reconstructed data; Calculate the cosine distance value of the covariance between the original data and the reconstructed data; The mean square error value and the cosine distance value are merged to obtain the reconstruction error.

5. The method for detecting abnormality of multi-dimensional time series data of a wind turbine according to claim 4, characterized in that: The reconstruction error satisfies: ; in, is the reconstruction error, is the mean square error between the time series and the reconstructed data, is the cosine similarity value; For control Hyperparameters of weights.

6. The method for detecting abnormality of multi-dimensional time series data of a wind turbine according to claim 5, characterized in that: Said Satisfied with: ; in, is the number of time points in the time series, is the number of variables in the time series, a represents the time point number, b represents the variable number, is the sample matrix of the original data, is the sample matrix of the reconstructed data; and / or Said Satisfied with: , ; in, Indicates the degree of similarity between two vectors in direction; represents cosine similarity; Represents the covariance of the original time series data; represents the covariance of the reconstructed data; represents the Euclidean norm; is the default value.

7. The method for detecting abnormality of multi-dimensional time series data of a wind turbine according to claim 4, characterized in that: The bidirectional long short-term network adopts an unsupervised learning strategy, uses the reconstruction error as the anomaly score, and uses a point adjustment strategy to adjust the anomaly label.

8. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Multi-dimensional time series data anomaly detection method

    CN116361635A

  • Time series data anomaly detection method based on multi-head attention model

    CN117076936A

  • Industrial time series data anomaly detection method based on space-time diagram attention network

    CN117272196A

  • Time series data prediction method associated with time-space relationship

    CN117668743A

  • Featured value extraction device for multi-dimensional time series data

    JP2008310520A