An industrial power data anomaly detection method based on multi-time scale trend perception attention

Abstract

The application discloses an industrial power data anomaly detection method based on multi-time scale trend perception attention, comprising the following steps: collecting industrial power time series data and preprocessing; constructing a time trend perception attention module based on a Transformer model; parallelly combining multiple time trend perception attention modules of different perception scales; extracting multi-scale context information of the preprocessed power time series data; cross-scale fusion of attention calculation results under each perception scale; outputting the predicted value of each power data point corresponding to the period; calculating the residual of the actual value and the predicted value; distributing the residual; determining the power data point as an abnormal point or a normal point according to the residual distribution analysis result. The application can accurately identify power sudden failures and hidden aging hazards, effectively solve the problem of missed judgment and misjudgment in manual investigation, improve the power anomaly investigation accuracy and efficiency, and ensure the stable operation of the industrial power system.

Description

Technical Field

[0001] This invention belongs to the field of industrial power technology, and in particular relates to a method for detecting anomalies in industrial power data based on multi-timescale trend-aware attention. Background Technology

[0002] Industrial power systems are the core support of intelligent manufacturing production systems, operating under complex conditions of high load and uninterrupted operation for extended periods. With the upgrading of industrial intelligence, power operation data exhibits massive, time-series-based, and highly volatile characteristics. The stability and safety of equipment operation directly determine the enterprise's production efficiency and safety level, making accurate power anomaly detection a core necessity for industrial operation and maintenance. Currently, mainstream industry operation and maintenance methods still rely on manual periodic inspections and traditional fixed-threshold detection modes, which are not only costly in terms of manpower and have poor timeliness but also extremely poor adaptability. Traditional detection methods cannot distinguish between normal fluctuations in production conditions and actual equipment faults, easily leading to misjudgments and missed diagnoses. Furthermore, existing detection technologies mostly employ single-time-scale data analysis models, making it difficult to address sudden faults such as instantaneous overloads and short circuits, as well as long-term hidden dangers such as line aging and performance degradation. This fails to achieve all-time, all-round operational condition monitoring, making it difficult to meet the high-precision, high-reliability intelligent operation and maintenance needs of modern industrial power systems. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes an industrial power data anomaly detection method based on multi-timescale trend-aware attention, thereby resolving the issues present in the prior art.

[0004] To achieve the above objectives, this invention provides a method for detecting anomalies in industrial power data based on multi-timescale trend-aware attention, comprising: Collect industrial power time-series data and preprocess it to obtain preprocessed industrial power time-series data; A time trend perception attention module is constructed based on the Transformer model; multiple time trend perception attention modules with different perception scales are combined in parallel; multi-scale context information is extracted from the preprocessed industrial power time series data through the combined multiple time trend perception attention modules; cross-scale information fusion is performed on the attention calculation results under each perception scale; and the predicted power data value for each power data point in the corresponding time period is output. Calculate the residual between the actual value and the predicted value of each power data point, perform distribution analysis on the residual, and determine whether the power data point is an anomaly or a normal point based on the residual distribution analysis results of each power data point, thus completing the anomaly detection of industrial power data.

[0005] Optionally, the process of collecting and preprocessing industrial power time-series data includes: The actual power data of industrial load equipment is continuously collected at a fixed sampling frequency to construct the original industrial power time series dataset. Missing data in the original industrial power time series data are filled in using neighborhood interpolation to obtain the preprocessed industrial power time series data.

[0006] Optionally, the multi-scale contextual information extraction of the preprocessed industrial power time-series data through multiple combined time-trend-aware attention modules includes: The preprocessed industrial power time series data is passed through three one-dimensional causal convolutional layers with different kernel sizes to extract multi-scale context information for each power data point. Based on the multi-scale context information, query matrix, key matrix and value matrix are generated for each perception scale. Based on the query matrix, the key matrix, and the value matrix at each perception scale, the attention matrix at each perception scale is calculated in a local contextual manner. The attention matrices at each perception scale are concatenated, and the multi-timescale trend perception calculation results of the power data are obtained based on the attention matrices at each perception scale.

[0007] Optionally, in the process of extracting multi-scale context information of each power data point through a one-dimensional causal convolutional layer, a masking mechanism is used to implement causal convolution, so that the convolution operation determines the multi-scale context information based on the current power data point and the power data points to its left.

[0008] Optionally, before concatenating the attention matrices at each perceptual scale, the method further includes: Each input power data point is assigned a location code, which is determined based on the power data point's position in the sequence and its feature dimension index.

[0009] Optionally, a distribution analysis is performed on the residuals, and the power data point is determined to be an outlier or a normal point based on the residual distribution analysis results, including: Arrange the residuals in ascending order to obtain an ordered residual sequence. The first and third quartiles of the residuals are determined based on the ordered residual sequence, and the interquartile range is determined based on the first and third quartiles. The upper and lower boundaries are determined based on the first and third quartiles, and the distribution boundaries of the residuals are determined based on the upper and lower boundaries. The residuals that exceed the distribution boundary are identified as abnormal residuals, and the power data points corresponding to the abnormal residuals are identified as abnormal points based on the abnormal residuals.

[0010] Optionally, determining the distribution boundary of the residual based on the upper boundary and the lower boundary includes: The lower boundary is defined as the first quartile minus 1.5 times the interquartile range, and the upper boundary is defined as the third quartile plus 1.5 times the interquartile range. Residuals that are less than the lower boundary or greater than the upper boundary are identified as abnormal residuals.

[0011] Optionally, the method also includes: Box plots are used to visualize and analyze the residuals. The box plots show the data distribution of normal and outlier points. The boxes of the box plots cover the first quartile to the third quartile. The whiskers outside the boxes extend to the minimum and maximum values ​​of the non-outlier values. Data points that exceed the extension range of the whiskers are marked as outliers.

[0012] Compared with the prior art, the present invention has the following advantages and technical effects: This invention integrates the basic Transformer model architecture and proposes an industrial power data anomaly detection method based on a multi-timescale trend-aware attention mechanism. By parallelly combining attention at different perception scales within the context, it synchronously captures multi-level features of the power sequence, enhancing the Transformer model's ability to represent complex power consumption patterns and fully integrating local contextual information. Through residual analysis using the IQR method between the actual value and the algorithm's predicted value, abnormal power data points are detected, and the power data analysis results are visualized using box plots. This invention can accurately identify sudden power failures and hidden aging hazards, effectively solving the problems of missed and misjudged detections during manual inspection, significantly improving the accuracy and efficiency of power anomaly detection, and ensuring the stable operation of industrial power systems. Attached Figure Description

[0013] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram illustrating the computational principle of the conventional attention mechanism in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the computational principle of the time trend-aware attention mechanism in an embodiment of the present invention. Figure 3 This is a multi-timescale trend-aware attention mechanism according to an embodiment of the present invention; Figure 4 Box plot comparing various methods for power data anomaly detection in embodiments of the present invention; Figure 5This is a schematic diagram of the power data anomaly detection results according to an embodiment of the present invention. Detailed Implementation

[0014] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0015] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0016] Example 1 This embodiment provides a method for detecting anomalies in industrial power data based on multi-timescale trend-aware attention, including: Step 1: Industrial power time-series data acquisition and preprocessing.

[0017] Step 2: Construct a time trend awareness attention module based on the Transformer model, improve the traditional attention mechanism in Transformer, and realize contextual trend awareness of industrial power data points.

[0018] Step 3: Parallel combination of time trend perception attention modules at different perception scales to further improve the Transformer, perceive the feature changes of power data at multiple levels, and output the predicted power data values ​​for the corresponding time period.

[0019] Step four: Using the residual between the actual power data and the predicted value from the algorithm, perform residual data analysis using the IQR method.

[0020] Step 5: Visualize the power data analysis results to complete the analysis and detection of abnormal industrial power data.

[0021] Step one is described in detail as follows: Actual power data from industrial load devices is continuously collected at a fixed sampling frequency to construct an original industrial power time-series dataset. To address the issue of missing power data due to sensor malfunctions, network fluctuations, and equipment start-up / shutdown interference in industrial settings, neighborhood interpolation is used to supplement the missing data, taking into account the continuous nature of industrial power time-series data. This ensures the integrity and continuity of the industrial power time-series data, providing effective and complete raw data support for subsequent anomaly detection.

[0022] Step two is described in detail below: Due to the influence of complex industrial production activities, industrial power data time series will undergo significant trend changes and exhibit non-stationary characteristics. Therefore, whether the attributes of power data points are outliers or components of normal sequence feature patterns depends heavily on the local contextual information of the power data.

[0023] However, the self-attention mechanism of the traditional Transformer model has significant limitations. Its query-key similarity calculation is based solely on point-by-point numerical comparison, failing to fully integrate local contextual information, such as... Figure 1 As shown. This design flaw may cause confusion in the self-attention module in critical scenarios. For example, when the power data at a certain time step becomes an outlier due to events such as equipment failure or load change, the Transformer model may incorrectly establish a strong correlation with data points in the normal pattern; or when the power data sequence undergoes a trend reversal, the model may misjudge the attributes of the power data points because it fails to capture the gradual change features in the neighborhood.

[0024] To solve the above problems, by constructing such Figure 2 The time-trend-aware attention mechanism shown can extract local contextual information for each power data point, enabling each node to perceive changes in contextual trends, thereby optimizing the traditional attention mechanism. Since constructing the time-trend-aware attention module in this step is the foundation of step three, to avoid repetition, the computational principles and process of the attention mechanism will be explained in detail in step three.

[0025] Step three is described in detail below: Temporal trend-aware attention can efficiently capture short-term dependencies and pattern features in power data sequences, perceiving trend changes within the context. However, single-scale context awareness often struggles to simultaneously address the needs of extracting power data features at different granularities. An excessively small context awareness scale may overemphasize local details of the power data while neglecting long-term trends, while an excessively large scale may dilute the information density of key time-series patterns. To overcome this challenge, this invention proposes a multi-time-scale trend-aware attention mechanism, such as... Figure 3 As shown, its core idea is to simultaneously capture multi-level features such as short-term fluctuations, medium-term cycles, and long-term trends in power data time series by combining attention at different perceptual scales in parallel. Based on the Transformer architecture and improved by applying the proposed attention mechanism, it enhances the Transformer model's ability to represent complex power consumption patterns. Through feature fusion, it achieves cross-scale information complementarity and enhancement, thus providing a more flexible feature extraction framework for anomaly detection in power data.

[0026] Specifically, for a given power data ZFirst, they are processed by convolution kernels with a size of 1× k 1, 1× k 2 and 1× k A 3D causal convolutional layer extracts multi-scale contextual information of power data points at each time step, generating... Q k and K k Generated through a 1×1 convolutional layer V k The calculation formula is as follows: In the formula: This indicates that the power data is processed by a convolution kernel of size 1× k 1D causal convolutional layer; This indicates that the power data is processed through a convolutional layer with a kernel size of 1×1.

[0027] Then reshape Q k , K k , V k Matrix, obtaining a multi-head representation of time-trend-aware attention: In the formula: This indicates that the original matrix is ​​reshaped using a multi-head approach; , and These are the multi-head representations of the original matrix after reshaping.

[0028] Subsequently, attention matrices for power data at each perception scale are generated using a local contextual approach. The calculation is performed using the following formula: In the formula: d k Indicates the dimensions of the query vector and key vector.

[0029] Since a multi-head representation reshaping operation was performed previously, the calculation results of multiple heads are reconstructed and concatenated again, and then processed using a weight matrix. W O Perform a linear transformation to obtain the final result A of the attention calculation for power data at each sensing scale. k The calculation formula is as follows: In the formula: This indicates a splicing and reshaping operation for multiple heads.

[0030] Finally, by combining the attention calculation results from different perceptual scales of the context, the cross-scale information of power data is complemented and enhanced: In the formula: This represents the concatenation operation of multiple time-scale trend-aware attention calculation results; A represents the calculation results of multi-time-scale trend awareness of power data.

[0031] In the multi-timescale trend-aware attention module of the decoder, a masking mechanism is needed to prevent the use of future power data information. This involves implementing 1D convolution of Q and K using causal convolution. By using causal convolution, it is ensured that only the left-hand information of the current power data point is considered, preventing the convolution operation from peeking into future power data information.

[0032] Furthermore, in the multi-timescale trend-aware attention module, to enable the model to utilize the location information of power data points, location encoding is introduced to assign location information to each power data element, making adjacent power data points have a closer location association. The formula for location encoding is: In the formula: t This indicates the position of the input power data point in the sequence; i The index representing the dimension of the positional encoding feature. d model Indicates the dimension of the feature vector.

[0033] This invention utilizes the anomaly detection concept of data prediction, integrates the basic Transformer model architecture, and achieves multi-scale capture of power data distribution characteristics through multi-timescale trend-aware attention, and outputs the predicted value of the time period to which the power data to be detected belongs.

[0034] Step four is described in detail below: The IQR method focuses on the main body of the data and has the ability to resist outlier interference. Outlier boundaries can be obtained through simple and efficient mathematical operations, thus enabling the detection of anomalies. Therefore, this invention utilizes the residual between the predicted and actual values ​​of power data obtained by a multi-timescale trend-aware attention algorithm, and then uses the IQR method to perform residual analysis on the power data, thereby identifying abnormal power data points.

[0035] First, calculate the residual between the actual and predicted values ​​of the power data. e i As shown in the following formula: In the formula: This represents the actual value of the electricity data; This represents the predicted value of the power data; n This represents the total number of power data items.

[0036] Next, all residuals e i Arranged in ascending order, we obtain an ordered residual sequence: In the formula: This represents the residual sample points sorted in ascending order.

[0037] The quartiles divide the ordered residual sequence of power data into four equal parts. The first quartile, Q1, and the third quartile, Q3, need to be calculated, where Q1 is the value located in the first 25% of the data, and Q3 is the value located in the first 75% of the data. The specific calculation formula is shown below: If the position is not an integer, it needs to be calculated by linear interpolation.

[0038] Then, the interquartile range (IQR) is calculated to reflect the dispersion of the data, as shown below: Based on the calculated IQR, the threshold range for abnormal residuals is defined as follows: In the formula: Lower Bound represents the lower boundary; Upper Bound represents the upper boundary.

[0039] Residuals that satisfy the following condition are considered outliers, and the corresponding samples in the original power data are potential outliers.

[0040] or Step five is described in detail below: This invention utilizes the IQR method to analyze the residuals of power data, enabling anomaly detection. To more intuitively display the data distribution of normal and outlier points, box plots are used for visual quantitative analysis of the power data residuals. The box plot, based on the IQR method, constructs boxes and visually displays the data distribution characteristics using five quantities: minimum, Q1, median, Q3, and maximum. In the box plot, the boxes cover Q1 to Q3, and the median line divides the boxes to reflect the central tendency of the data. The "whiskers" outside the boxes extend to the minimum / maximum values ​​of non-outliers, i.e., the anomaly boundary; data points exceeding this boundary are marked as outliers. The box plot reveals the data distribution and outliers, providing a complete analytical chain from statistical calculation to graphical display for anomaly detection in power data.

[0041] The effects of the present invention are further illustrated below with application examples: The official open-source database of the U.S. Energy Information Administration recorded industrial electricity consumption data for a certain area in the heavy industrial region of northeastern United States from March 1 to March 30, 2017. Electricity consumption was counted every 10 minutes. The electricity dataset was used for comparative analysis of multiple methods for anomaly detection. The data from March 30 was used to test the performance of each method, and the rest was used for model training and validation. The experiment used multiple evaluation indicators to evaluate the results of anomaly detection: (1) False Positive Rate (FPR), which refers to the proportion of normal samples that are misclassified as anomalies, reflecting the degree of misjudgment of normal samples by the model. (2) False Negative Rate (FNR), which refers to the proportion of anomalous samples that are misclassified as normal, reflecting the degree of missed detection of anomalous samples by the model. (3) Accuracy, which is the proportion of all samples that are correctly classified, and it comprehensively measures the overall performance of the model. The calculation formulas for the above three evaluation indicators are defined as follows: In the formula: FP represents the number of samples that are actually normal but are predicted as abnormal; TN represents the number of samples that are actually normal and are correctly predicted as normal; FN represents the number of samples that are actually abnormal but are predicted as normal; TP represents the number of samples that are actually abnormal and are correctly predicted as abnormal.

[0042] Experimental results are as follows Figure 4 , Figure 5 As shown in Table 1. Figure 4The box plot, based on the residuals between actual and predicted power data, illustrates the residual distribution of power data under various methods. The box plot shows that the iTransformer method can detect 17 anomalous power data points, the Pyraformer and M-GTU methods can each detect 18 anomalous power data points, while the method proposed in this invention can detect 19 anomalous power data points. Figure 5 The power data anomaly detection results obtained by the method proposed in this invention are relatively smooth and close to the normal data distribution. For the two sudden abnormal power data points, both can be accurately detected. For the three sets of continuous anomalies, only three abnormal power data points were not correctly detected.

[0043] Analyzing the evaluation indicators in Table 1, for all methods in the experiment, the false alarm rate for power data anomaly detection was 0, indicating that no normal power data points were mistakenly identified as anomalies. The iTransformer method performed poorly in anomaly detection, with a false negative rate of 22.73%, which was 66.64% higher than the method proposed in this invention, and an accuracy of 96.53%, which was 1.42% lower than the method proposed in this invention. The Pyraformer and M-GTU methods had the same false negative rate and accuracy, but the false negative rate was still 33.28% higher than the method proposed in this invention, and the accuracy was 0.71% lower. In summary, for power data anomaly detection, the method proposed in this invention is superior to the three comparative methods. This anomaly detection method can improve the quality of industrial power data, support power data analysis and strategy formulation, and ensure the safe and stable operation of industrial power systems.

[0044] Table 1 The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting anomalies in industrial power data based on multi-timescale trend-aware attention, characterized in that, Includes the following steps: Collect industrial power time-series data and preprocess it to obtain preprocessed industrial power time-series data; A time-trend-aware attention module is constructed based on the Transformer model; Multiple time-trend perception attention modules with different perception scales are combined in parallel. The combined time-trend perception attention modules are used to extract multi-scale context information from the preprocessed industrial power time series data. The attention calculation results at each perception scale are fused across scales to output the predicted power data value for each power data point in the corresponding time period. Calculate the residual between the actual value and the predicted value of each power data point, perform distribution analysis on the residual, and determine whether the power data point is an anomaly or a normal point based on the residual distribution analysis results of each power data point, thus completing the anomaly detection of industrial power data.

2. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 1, characterized in that, The process of collecting and preprocessing industrial power time-series data includes: The actual power data of industrial load equipment is continuously collected at a fixed sampling frequency to construct the original industrial power time series dataset. Missing data in the original industrial power time series data are filled in using neighborhood interpolation to obtain the preprocessed industrial power time series data.

3. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 1, characterized in that, Multi-scale contextual information extraction is performed on the preprocessed industrial power time-series data using multiple combined time-trend-aware attention modules, including: The preprocessed industrial power time series data is passed through three one-dimensional causal convolutional layers with different kernel sizes to extract multi-scale context information for each power data point. Based on the multi-scale context information, query matrix, key matrix and value matrix are generated for each perception scale. Based on the query matrix, the key matrix, and the value matrix at each perception scale, the attention matrix at each perception scale is calculated in a local contextual manner. The attention matrices at each perception scale are concatenated, and the multi-timescale trend perception calculation results of the power data are obtained based on the attention matrices at each perception scale.

4. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 1, characterized in that, In the process of extracting multi-scale context information for each power data point through a one-dimensional causal convolutional layer, a masking mechanism is used to implement causal convolution, so that the convolution operation determines the multi-scale context information based on the current power data point and the power data points to its left.

5. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 1, characterized in that, Before concatenating the attention matrices at each perceptual scale, the following steps are also included: Each input power data point is assigned a location code, which is determined based on the power data point's position in the sequence and its feature dimension index.

6. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 1, characterized in that, The residuals are subjected to distribution analysis. Based on the residual distribution analysis results of each power data point, the power data point is determined to be an outlier or a normal point, including: Arrange the residuals in ascending order to obtain an ordered residual sequence. The first and third quartiles of the residuals are determined based on the ordered residual sequence, and the interquartile range is determined based on the first and third quartiles. The upper and lower boundaries are determined based on the first and third quartiles, and the distribution boundaries of the residuals are determined based on the upper and lower boundaries. The residuals that exceed the distribution boundary are identified as abnormal residuals, and the power data points corresponding to the abnormal residuals are identified as abnormal points based on the abnormal residuals.

7. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 6, characterized in that, Determining the distribution boundary of the residual based on the upper boundary and the lower boundary includes: The lower boundary is defined as the first quartile minus 1.5 times the interquartile range, and the upper boundary is defined as the third quartile plus 1.5 times the interquartile range. Residuals that are less than the lower boundary or greater than the upper boundary are identified as abnormal residuals.

8. The industrial power data anomaly detection method based on multi-timescale trend-aware attention as described in claim 1, characterized in that, The method also includes: Box plots are used to visualize and analyze the residuals. The box plots show the data distribution of normal and outlier points. The boxes of the box plots cover the first quartile to the third quartile. The whiskers outside the boxes extend to the minimum and maximum values ​​of the non-outlier values. Data points that exceed the extension range of the whiskers are marked as outliers.

Images