Time series data anomaly detection and quality evaluation method and system fusing attention mechanism and statistical constraint
By combining variational model decomposition and statistical constraints with deep learning, the accuracy problem of segment anomaly detection in time series data was solved, and adaptive threshold setting and efficient quality assessment were achieved.
Patent Information
- Application Number
- CN202510907440.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-02
- Publication Date
- 2025-11-04
AI Technical Summary
Existing deep learning-based time-series data anomaly detection methods perform poorly on segment anomalies and are difficult to adaptively set anomaly thresholds.
We employ variational model decomposition for data augmentation, combine statistical constraints and deep learning methods, adaptively find the optimal threshold using a small sample dataset and energy valley optimization algorithm, and integrate an attention mechanism for anomaly detection and quality assessment.
It improves the accuracy of fragment anomaly detection, solves the problem of difficult threshold setting, and performs well on non-stationary data.
Smart Images
Figure CN120892702A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of time series data quality evaluation, and particularly relates to a time series data anomaly detection and quality evaluation method, system, device and storage medium fusing an attention mechanism and statistical constraints. BACKGROUND
[0002] Time series data widely exist in the fields of finance, power grid, meteorology, etc., and correct time series data provides a solid foundation for subsequent financial market analysis, meteorological prediction and other tasks. However, due to various reasons, noise is easily introduced when collecting time series data. For example, in the field of meteorology, sensor failure, signal interference and other reasons can cause abnormal monitoring meteorological data to be received. If the proportion of abnormal data is large, and the data is not controlled in quality, it is likely to affect the accuracy of subsequent meteorological prediction and other tasks, so it is necessary to perform anomaly detection and quality evaluation on the collected time series data.
[0003] The common anomalies of time series data at present include point anomaly, segment anomaly and variable correlation anomaly. Point anomaly refers to the mutation of data at a certain time point in the time series, and this anomaly is manifested as a single point deviating from the global or local normal data mode. Segment anomaly refers to the situation that a continuous data point in the time series deviates from the normal data mode, which is usually manifested as a sudden change in data periodicity, trend or only a shift in the overall waveform. Variable correlation anomaly refers to the situation that for multivariate time series data, there is a feature with a large correlation with the detected variable, and at this time, the correlation between variables may change, thereby causing an anomaly.
[0004] For the above three kinds of anomalies, researchers have proposed many anomaly detection algorithms, which can be divided into three categories, namely statistical-based methods, machine learning-based methods and deep learning-based methods. Among them, the statistical-based method is the earliest method, which mostly finds outliers by setting corresponding rules such as limit value, difference value, etc., but this way is prone to misjudgment. The machine learning-based method alleviates the problems existing in the statistical method to some extent, but they are often limited by feature correlation. For example, regression methods such as random forest, XGBoost, etc., which depend on features with strong correlation with the detected variable, which are often difficult to obtain. With the advent of deep learning-based methods, these problems have been solved. For example, variational autoencoder (VAE) can learn the underlying distribution of clean data to reconstruct data conforming to the mode and thus realize anomaly detection, which performs well on point anomaly.
[0005] However, the existing deep learning-based method still has two problems, the first is that when a segment anomaly occurs, especially a large area of segment anomaly, the underlying distribution and pattern of the original data may be changed, making it difficult to reconstruct the expected data, and the second is that the setting of the anomaly threshold depends on artificial experience or complex parameter tuning. SUMMARY
[0006] The purpose of the present application is to provide a time series data anomaly detection and quality evaluation method, system, device and storage medium that fuses attention mechanism and statistical constraints, which can solve the problem that the existing deep learning-based method performs poorly on segment anomalies.
[0007] Technical scheme: The time series data anomaly detection and quality evaluation method fusing attention mechanism and statistical constraints comprises:
[0008] Offline access to high-quality time series data, and perform data preprocessing on the high-quality time series data to obtain clean training data;
[0009] Establish an initial time series data autoregressive model; input the training data into the initial time series data autoregressive model, and perform back propagation offline update of the network parameters of the initial time series data autoregressive model; after repeating multiple times, a trained time series data autoregressive model is obtained;
[0010] Using the EVO algorithm and the trained time series data autoregressive model on the labeled small sample data set, the threshold is adaptively optimized to obtain the optimal threshold;
[0011] Input the time series data to be detected into the trained time series data autoregressive model, judge whether the mean square error of the calculation result of the trained time series data autoregressive model and the time series data to be detected exceeds the optimal threshold, and fuse the statistical constraints to output the final anomaly detection and quality evaluation result.
[0012] Further, offline access to high-quality time series data, and perform data preprocessing on the high-quality time series data to obtain clean training data, comprising:
[0013] Offline access to high-quality multivariate time series data of multiple years, and integrate and save all multivariate time series data into a CSV file;
[0014] Read the CSV file, sort by time and interpolate to supplement missing timestamps, remove outliers in the multivariate time series data, and fill in missing values according to the average value;
[0015] Using a variational model decomposition to split the variable to be detected into high-frequency components, medium-frequency components and low-frequency components, thereby performing data enhancement on the original variable;
[0016] All frequency domain components are spliced with the rest of the variables of the multivariate time series data by column to construct clean training data.
[0017] Further, the variable to be detected is split into high-frequency components, medium-frequency components, and low-frequency components using a variational model decomposition, thereby performing data enhancement on the original variable, including:
[0018] Construct a variational constraint model: assuming that the detection variable f is decomposed into k components, given the constraint condition that the sum of all modes is equal to the original signal, the expression of the variational constraint model of VMD is as follows:
[0019]
[0020] where u k ={u1,u2,…,u K} is each modal function; ω k ={ω1,ω2,…,ω K} is the center frequency of each mode; The essence is to perform a Hilbert transform on u k ; and
[0021] Solve the variational problem: according to the objective function and the constraint condition of the variational constraint model, define the Lagrange function; perform Fourier transform on all variables to the frequency domain; by repeatedly updating each mode and its center frequency, until convergence or reaching the upper limit of iteration, each u k can be obtained; and the inverse Fourier transform is performed to return to the time domain.
[0022] Further, the training data is input into an initial time series data autoregressive model, and the initial time series data autoregressive model is updated offline by back propagation; after repeating multiple times, a trained time series data autoregressive model is obtained, including:
[0023] The clean training data x is input, down-sampled by average pooling AvgPooling(·) to obtain multi-scale time series data X={x0,…,x M}, where M is the number of scales, and projected into the first high-dimensional feature X={x0,…,x M} by embedding Embed(·), and the process is as follows:
[0024] X=AvgPooling(x)
[0025] X=Embed(X)
[0026] A plurality of stacked sequence fusion modules are set to mix information of the first high-dimensional feature X, and extract the bottom mode of the mixed first high-dimensional feature X; for the lth sequence fusion module TSFMl (·), input is X l-1 , output X l Can be expressed as:
[0027] X l = TSFM l (X l-1 )
[0028] After L stacked sequence fusion modules, the output of the sequence fusion module is
[0029] The pre-designed feature extractor Extractor(·) is used to extract deeper value information from clean training data x, and project it into second high-dimensional features
[0030]
[0031] Will Input the predictor Prediction respectively according to the scale m (·), and cross attention CrossAttention (·) operation with the second high-dimensional features m , where m∈{0,…,M}, get the output of the attention submodule
[0032]
[0033] The calculation formula of cross attention is:
[0034]
[0035] Where Q, K, V represent the query matrix, key matrix and value matrix respectively; d k Indicates the dimension of the key matrix; T represents transposition.
[0036] The output of the attention submodule Input the regressor Regression m (·) according to the scale, and accumulate to get the component reconstructed by the time series data autoregressive model
[0037]
[0038] The above steps are repeated to perform parameter optimization and weight update with MSE as the loss function; the learning rate is dynamically adjusted by the Adam optimizer and the OneCycleLR strategy to achieve better optimization effect on the model parameters, and finally the optimal network parameters are output. The optimal network parameters are imported into the initial time series data autoregressive model to obtain the trained time series data autoregressive model.
[0039] Further, the first sequence fusion module TSFM l The workflow of (·) is as follows:
[0040] In the first sequence fusion module, the first high-dimensional feature X is subjected to multi-scale sequence decomposition SeriesDecomp(·). For each scale m∈{0,…,M}, the seasonal term and the trend term can be expressed as:
[0041]
[0042] where, is the hidden variable to be decomposed;
[0043] Further, the multi-scale seasonal term and the trend term
[0044] The multi-scale trend term is subjected to trend term mixing TrendMix(·), and the mixed trend term is added to the seasonal term of the finest scale to obtain:
[0045]
[0046] Therefore, the output of the first sequence fusion module is
[0047] Further, the EVO algorithm and the trained time series data autoregressive model are used on the labeled small sample data set to perform adaptive optimization of the threshold value, and the optimal threshold value is obtained, including:
[0048] For the variable to be detected, a small sample data set containing multiple anomalies is constructed, and is manually labeled;
[0049] The constructed labeled small sample data set is subjected to VMD decomposition and splicing and then input into the trained time series data autoregressive model to obtain reconstructed components;
[0050] The reconstructed components are added column by column to obtain reconstructed data of the detected variable, and the mean square error MSE of the reconstructed data of the detected variable and the original data of the detected variable is calculated;
[0051] The lower limit and upper limit of the mean square error threshold are artificially set, the threshold is initialized, the abnormal detection result is obtained by comparing the mean square error, and the F1_Score is calculated according to the abnormal detection result and the manually labeled label;
[0052] The energy valley optimization algorithm is introduced, the F1_Score is taken as a fitting function, and the threshold corresponding to the highest F1_Score is found through continuous iteration, and the threshold at this time can be regarded as the optimal threshold.
[0053] Further, the time series data to be detected is input into the trained time series data autoregressive model, whether the calculation result of the trained time series data autoregressive model exceeds the optimal threshold is judged, and the final abnormal detection and quality evaluation result is output by fusing statistical constraints, including:
[0054] Accessing the time series data to be detected, the time series data to be detected is preprocessed;
[0055] The preprocessed time series data is input into the trained time series data autoregressive model, the reconstructed components are regressed, all the reconstructed components are added to obtain the reconstructed data, the MSE of the reconstructed data and the original data of the detection variable is calculated to obtain the deviation, and the deviation is compared with the optimal threshold, if the deviation is greater than the threshold, it is abnormal, otherwise it is normal, and the abnormal detection result is output according to the judgment;
[0056] Statistical constraints are introduced, limit value setting and first-order difference judgment are performed on the preprocessed time series data, and the abnormal detection result of the statistical method is obtained;
[0057] The preliminary detection result is obtained by comprehensively combining the abnormal detection result of the deep learning and the abnormal detection result of the statistics; the final abnormal detection result is obtained by comparing the corrected value with the original time series data according to the preliminary detection result; and the quality of the time series data is finally evaluated according to the proportion of abnormal values in the detected time series data.
[0058] Based on the same inventive concept, a time series data anomaly detection and quality evaluation system fusing attention mechanism and statistical constraints, comprising:
[0059] The data preprocessing module is used for offline accessing high-quality time series data, and pre-processing the high-quality time series data to obtain clean training data;
[0060] The time series autoregressive module is used for establishing an initial time series data autoregressive model; the training data is input into the initial time series data autoregressive model, and the initial time series data autoregressive model is updated offline by back propagation; after repeated several times, the trained time series data autoregressive model is obtained;
[0061] An adaptive threshold calculation module is configured to perform adaptive optimization of the threshold using the EVO algorithm and the trained time series data autoregressive model on the labeled small sample dataset to obtain the optimal threshold.
[0062] An anomaly detection and evaluation module is configured to input the time series data to be detected into the trained time series data autoregressive model, judge whether the mean square error of the calculation result of the trained time series data autoregressive model and the time series data to be detected exceeds the optimal threshold, and output the final anomaly detection and quality evaluation result by fusing the statistical constraint.
[0063] Based on the same inventive concept, the present application provides a time series data anomaly detection and quality evaluation device fusing an attention mechanism and a statistical constraint, which comprises a processor and a memory, wherein the memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device implements the steps of the time series data anomaly detection and quality evaluation method fusing the attention mechanism and the statistical constraint.
[0064] Based on the same inventive concept, the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the time series data anomaly detection and quality evaluation method fusing the attention mechanism and the statistical constraint are implemented.
[0065] Advantages: Compared with the prior art, the present application has the following significant technical effects:
[0066] The present application enhances the stability of the data by performing data enhancement on the detection variables through VMD, and can remove large noise to retain the underlying distribution and pattern of the data.
[0067] In the present application, not only the deep learning method is used, but also the statistical constraint is introduced, the advantages of both are taken into account, and the detection accuracy on the segment anomaly is improved.
[0068] In the present application, the threshold value with the best anomaly detection effect is found through the use of the labeled small sample dataset and the energy valley optimization algorithm (EVO) in continuous iteration, which is convenient for subsequent detection, and solves the problem of constantly adjusting the threshold value parameter by human in the prior art.
[0069] The present application not only can perform anomaly detection, but also can correct the abnormal data according to the underlying distribution and pattern of the data, and the corrected data is reasonable and usable.
[0070] The present application is not only suitable for periodic and stationary data, but also performs well on non-stationary data, which is due to the powerful processing capability of the variational model decomposition (VMD) on non-stationary data. Attached Figure Description
[0071] Figure 1 This is a flowchart illustrating a time-series data anomaly detection and quality assessment method that integrates attention mechanisms and statistical constraints, as disclosed in an embodiment of the present invention.
[0072] Figure 2 This is a schematic diagram of the structure of a time-series data anomaly detection and quality assessment system that integrates attention mechanisms and statistical constraints, as disclosed in an embodiment of the present invention.
[0073] Figure 3 This is a schematic diagram of the structure of a time-series data anomaly detection and quality assessment device that integrates attention mechanisms and statistical constraints, as disclosed in an embodiment of the present invention. Detailed Implementation
[0074] The technical solution of the present invention will now be described in detail with reference to specific embodiments and accompanying drawings.
[0075] Example 1
[0076] Please see Figure 1 , Figure 1 This is a flowchart illustrating a time-series data anomaly detection and quality assessment method that integrates attention mechanisms and statistical constraints, as disclosed in an embodiment of the present invention. Figure 1 The described time-series data anomaly detection and quality assessment method is used for time-series data quality assessment, such as for anomaly detection in time-series data, and is not limited to this purpose in the embodiments of the present invention. Figure 1 As shown, this method for anomaly detection and quality assessment of time-series data, which integrates attention mechanisms and statistical constraints, may include the following operations:
[0077] S1. Access high-quality time-series data offline and perform data preprocessing on the high-quality time-series data to obtain clean training data.
[0078] In step S1, data access, data cleaning, and data augmentation are completed according to steps S1.1 to S1.4, based on the requirements of the training data, and clean training data is output. The specific implementation process of step S1 is as follows:
[0079] S1.1: Offline access to high-quality multivariate time series data from multiple years, and integration and saving of all multivariate time series data into a CSV file.
[0080] High-quality multivariate time series data from multiple years should be completely clean or have very few anomalies. The data should be concatenated row by row and saved locally as a CSV file.
[0081] S1.2, read the CSV file, sort by time and interpolate to fill in missing timestamps, remove outliers in multivariate time series data, and fill in missing values with the average value.
[0082] Read the CSV file, as the data volume is large when downloading, it is easy to appear time disorder. Sort the data by time, and find that some timestamps are missing after sorting. Reset the timestamp and fill the data to the corresponding position. At this time, there are very few outliers and missing values in the data, in order to ensure that the subsequent autoregressive model training is not affected, fill in the average value of the last five values.
[0083] S1.3, use the variational model decomposition (VMD) to split the variable to be detected into high-frequency components, medium-frequency components and low-frequency components, thereby enhancing the data of the original variable; the high-frequency information contains short-time change information, the medium-frequency information contains periodic change, and the low-frequency information contains long-term trend.
[0084] In this embodiment, the specific implementation process of step S1.3 is as follows:
[0085] S1.3.1, construct a variational constraint model: assume that the detection variable f is decomposed into k components, in order to ensure that the decomposition sequence is a modal component with a central frequency and a limited bandwidth, and the sum of the estimated bandwidths of each mode is minimized, the given constraint condition is that the sum of all modes is equal to the original signal, then the expression of the variational constraint model of VMD is as follows:
[0086]
[0087] Where, u k ={u1,u2,…,u K} is each modal function; ω k ={ω1,ω2,…,ω K} is the center frequency of each mode; The essence is to perform Hilbert transform on u k .
[0088] Hilbert transform can be regarded as passing the original signal through a filter. Compared with the convolution concept, it is actually the result of convolving the original signal with the filter signal.
[0089] S1.3.2, solve the variational problem: according to the objective function and constraint condition of the variational constraint model, first define the Lagrange function; then Fourier transform all variables to the frequency domain; then update each mode and its center frequency by repeatedly alternating until convergence or the iteration limit is reached, so that each u k can be obtained; finally, the inverse Fourier transform is converted back to the time domain.
[0090] The specific implementation process of step S1.3.2 is as follows:
[0091] According to the objective function of the variational constraint model and the constraint condition, first define the Lagrange function:
[0092]
[0093] Where λ(t) is the Lagrange multiplier; then Fourier transform u k (t), f(t), λ(t) into frequency domain representation, respectively And Then by repeatedly updating each mode and its center frequency, until convergence or reaching the iteration upper limit, each u k In each iteration, for the k-th mode, assuming that the remaining modes are known and fixed, first update
[0094]
[0095] Where α is the penalty term coefficient, and then calculate the new center frequency:
[0096]
[0097] And update the Lagrange multiplier:
[0098]
[0099] Where τ is the step size, used to control the update speed; finally, inverse Fourier transform is used to convert back to the time domain.
[0100] S1.4, splice all the frequency domain components obtained in step S1.3 with the remaining variables of the multivariate time series data by column to replace the original detection variables, and construct clean training data.
[0101] Through data cleaning and variational model decomposition (VMD) on the accessed time series data, the data enhancement of the detection variables is realized, and the accuracy of subsequent anomaly detection is improved.
[0102] S2, establish an initial time series data autoregressive model; input the training data in step S1 into the initial time series data autoregressive model, and perform back propagation offline update of network parameters on the initial time series data autoregressive model; after repeating several times, a trained time series data autoregressive model is obtained.
[0103] Where the training data is input into the initial time series data autoregressive model, and the initial time series data autoregressive model is updated offline by back propagation; after repeating several times, a trained time series data autoregressive model is obtained, which specifically includes the following steps:
[0104] S2.1, input clean training data x, down-sample by average pooling AvgPooling(·) to get multi-scale time series data X = {x0, …, xM}, where M is the number of scales, and project it to the first high-dimensional feature X = {x0, …, xM} by embedding layer Embed(·), the process is as follows: M M
[0105] X = AvgPooling(x)
[0106] X = Embed(X)
[0107] S2.2, set a plurality of stacked sequence fusion modules (TSFM), mix the information of the first high-dimensional feature X in step S2.1, and extract the underlying patterns of the high-dimensional data in the mixed first high-dimensional feature X. For the lth sequence fusion module TSFM l (·), the input is X l-1 , and the output X l can be represented as:
[0108] X l = TSFM l (X l-1 )
[0109] After L stacked sequence fusion modules, the output of the sequence fusion module is
[0110] In this embodiment, the workflow of the lth sequence fusion module TSFM l (·) is as follows:
[0111] S2.2.1, in the lth sequence fusion module, the first high-dimensional feature X in step S2.1 is subjected to multi-scale sequence decomposition SeriesDecomp(·), the sequence decomposition uses the sequence decomposition unit used in the AutoTransformer paper, which is based on the idea of moving average, smoothing the periodic term and highlighting the trend term. For each scale m ∈ {0, …, M}, the seasonal term and the trend term can be represented as:
[0112]
[0113] More specifically:
[0114]
[0115] where, is the hidden variable to be decomposed; and are the trend and seasonality terms, respectively. For moving average, padding operation is applied to keep the sequence length unchanged.
[0116] S2.2.2, after step S2.2.1, the multi-scale seasonality terms can be obtained and the trend terms
[0117] S2.2.3, the multi-scale seasonality terms obtained in step S2.2.2 and the multi-scale trend terms the finest scale seasonality terms are taken out respectively and the trend term mixing TrendMix(·) is performed, because when performing time series anomaly detection, the fine seasonality terms are more worthy of attention. And the mixed trend terms are added to obtain
[0118]
[0119] Therefore, the output of the lth sequence fusion module is
[0120] S2.3, a pre-designed feature extractor Extractor(·) is used to extract deeper value information from clean training data x, and project it into second high-dimensional features
[0121]
[0122] S2.4, the output of step S2.2 is input into the predictor Prediction m (·) by scale respectively, and cross attention CrossAttention m (·) operation is performed with the second high-dimensional features in step S2.3 , where m∈{0,…,M}, to obtain the output of the attention submodule This way can extract the periodicity and trend of the data in S2.2 while weighting the value features to prevent the over-fading of the value features. The output of the attention submodule is expressed as follows:
[0123]
[0124] The calculation formula of cross attention is:
[0125]
[0126] where Q, K, V represent query matrix, key matrix and value matrix respectively; d k denotes the dimension of key matrix; T denotes transpose. First, the dot product of query and key is calculated, which represents the similarity of two sequences at different positions; then d k is taken as a scaling factor to avoid too large values; then the softmax function is used to convert these similarities into probability distribution, which represents the attention weight of query to each key; finally, these attention weights are applied to the value V to obtain the output vector. This is equivalent to extracting the information of interest from the past feature sequence and inputting it into the regressor.
[0127] At this time, Q, K, V can be expressed as:
[0128] S2.5, the output of the attention sub-module in step S2.4 is scaled and input into the regressor Regression m (·) and accumulated to obtain the component reconstructed by the time series data autoregressive model
[0129]
[0130] The above steps S2.1-S2.5 are repeated, and the MSE is used as the loss function for parameter optimization and weight update. Through the Adam optimizer and using the OneCycleLR strategy to dynamically adjust the learning rate, a better optimization effect is achieved for the model parameters, and finally the optimal network parameters are output. The optimal network parameters are imported into the initial time series data autoregressive model to obtain the trained time series data autoregressive model.
[0131] After each epoch of training, the MSE is calculated using the validation set. If the result of the last 10 times is not better than the previous one, the training is stopped in advance, and the network parameters corresponding to the best result are output.
[0132] In step S2, for the network parameter training required, the offline training of the time series data autoregressive model is completed according to the above S2.1-S2.6.
[0133] By inputting the preprocessed data, the data conforming to the pattern is reconstructed from the data containing anomalies according to the underlying distribution and pattern of clean data learned in the training phase, and the mean square error (MSE) with the original data is calculated.
[0134] S3, using the EVO algorithm and the trained time series data autoregressive model in step S2 on the labeled small sample data set to perform adaptive optimization of the threshold value, and obtaining the optimal threshold value.
[0135] In step S3, to solve the problem of threshold setting, the calculation of the adaptive threshold is completed according to S3.1-S3.5:
[0136] S3.1, for the variable to be detected, a small sample data set containing multiple anomalies (such as data point anomalies, segment anomalies, etc.) is constructed, and is manually labeled, and the anomalies in the constructed small sample data set are ensured to conform to the distribution of the detected anomalies;
[0137] S3.2, the labeled small sample data set constructed is input into the trained time series data autoregressive model after VMD decomposition and splicing according to steps S1.3 and S1.4, to obtain reconstructed components;
[0138] S3.3, the reconstructed components are added column by column to obtain the reconstructed data of the detection variable, and the mean square error (MSE) of the reconstructed data of the detection variable and the original data of the detection variable is calculated, and the calculation formula of the MSE is:
[0139]
[0140] Wherein, y i is the original data, is the reconstructed data. The MSE can better measure the difference between them.
[0141] S3.4, the lower limit and the upper limit of the mean square error threshold are set manually, the threshold is initialized first, and a value is randomly selected from the upper and lower limits as the initial iteration threshold; then the anomaly detection result is obtained by comparing the initial iteration threshold with the mean square error MSE in step S3.3, if the MSE is greater than the initial iteration threshold, it is an anomaly, otherwise it is normal data; finally, the anomaly detection result is converted to 0, 1 value and the F1_Score is calculated with the manually labeled label, and the calculation formula of the F1_Score is as follows:
[0142]
[0143] Wherein, precision is the precision, which refers to the proportion of samples whose predicted value is 1 and whose true value is also 1 in all samples whose predicted value is 1. Recall is the recall, also known as the recall rate, which refers to the proportion of samples whose predicted value is 1 and whose true value is also 1 in all samples whose true value is 1. In other words, the precision measures which of the detected anomalies are real anomalies, and the recall measures how many real anomalies are found. Therefore, the F1_Score is more suitable for measuring the effect of anomaly detection.
[0144] S3.5, introduce energy valley optimization algorithm (EVO), take F1_Score as fitting function, through continuous iteration, until the threshold value corresponding to the highest F1_Score is found, at this time, the threshold value can be regarded as the optimal threshold value.
[0145] The iteration process of EVO in step S3.5 is as follows:
[0146] S3.5.1, randomly generate an initial particle swarm, each particle represents a threshold value. And make hyperparameter setting, including particle number N, maximum iteration number T, energy attenuation factor β and energy field gradient learning rate α.
[0147] S3.5.2, for each particle (threshold value), calculate F1_Score using the anomaly detection result and human-labeled label, define energy E i = 1-F1, the goal is to minimize energy.
[0148] S3.5.3, calculate the energy gradient of the current particle And update the particle position:
[0149]
[0150] Where, α represents the learning rate, used to control the step size of gradient descent; β represents the energy attenuation factor, used to introduce random disturbance, to avoid falling into local optimum; rand() represents random disturbance.
[0151] S3.5.3, repeat S3.5.2 and S3.5.3 until the maximum iteration number T is reached or the energy converges, the particle corresponding to the lowest energy is the optimal threshold value.
[0152] By inputting small sample labeled data into the time series autoregressive model, and introducing energy valley optimization algorithm (EVO), through multiple iterations, find the best threshold value on the data set.
[0153] S4, input the time series data to be detected into the time series data autoregressive model trained in step S2, judge whether the mean square error of the calculation result of the trained time series data autoregressive model and the time series data to be detected exceeds the optimal threshold value in step S3, and fuse statistical constraints to output the final anomaly detection and quality evaluation results.
[0154] In step S4, for the quality evaluation of the detection data, complete the anomaly detection of the data according to S4.1-S4.4:
[0155] S4.1, access the time series data to be detected, and perform data preprocessing on the time series data to be detected. Unlike the training phase, since the target of the inference phase is anomaly detection, only timestamp supplementing and 0 filling of missing values are performed. Subsequent VMD decomposition and splicing operations are consistent with the training phase, so as to obtain the input data of the inference phase.
[0156] S4.2, input the preprocessed time series data into the trained time series data autoregressive model, regress the reconstructed components, add all the reconstructed components to obtain the reconstructed data. Calculate the MSE of the reconstructed data and the original data of the detection variable, obtain the deviation, and compare the deviation with the optimal threshold value in S3.5. If the deviation is greater than the threshold value, it is abnormal, otherwise it is normal. Output the anomaly detection result according to the judgment.
[0157] S4.3, introduce statistical constraints, set the limit value of the preprocessed time series data, and judge the first-order difference, so as to obtain the anomaly detection result of the statistical method.
[0158] In this embodiment, statistical constraints are introduced, the limit value of the preprocessed time series data is set, and greater than the maximum limit or less than the minimum limit is regarded as abnormal. At the same time, the first-order difference feature of the detection variable is added, which is used to detect the situation of large-scale and violent change. If the difference value is greater than 3 times the optimal threshold value, it is regarded as abnormal. For abnormal values, the way of correction in the frequency domain is adopted, which can effectively solve the problem of segment anomaly. The statistical feature can assist the anomaly detection of the deep learning method.
[0159] S4.4, first, the anomaly detection results of deep learning and the anomaly detection results of statistics are combined to obtain the preliminary detection results; then, the preliminary detection results are further corrected, and the corrected values are compared with the original time series data to obtain the final anomaly detection results; finally, according to the proportion of abnormal values in the detected time series data, the final quality of the time series data is evaluated.
[0160] In this embodiment, first, the anomaly detection results of deep learning and the anomaly detection results of statistics are combined to obtain the preliminary anomaly detection results; then, the preliminary detection results are further corrected, and the corrected values are compared with the original data to obtain the final anomaly detection results; finally, according to the proportion of abnormal values in the data, the final quality of the data is evaluated.
[0161] By comparing the output of the time series autoregressive model with the adaptive threshold value, and simultaneously combining the auxiliary detection features obtained by introducing statistical constraints, the final anomaly detection result is obtained, and the data is corrected according to the result to make the data conform to the normal mode.
[0162] In the present application, first, data cleaning is performed, and the variable to be detected is decomposed into multiple frequency domain components to construct input data using VMD; then, a time series data autoregressive model is established, and the network parameters are trained offline using back propagation using clean data to learn the underlying distribution and pattern of clean data; then, the trained time series data autoregressive model and EVO algorithm are used to adaptively optimize the threshold on the labeled small sample data set; finally, the data to be detected is input into the model, and the underlying pattern of clean data is reconstructed according to the data, and whether the threshold is exceeded is calculated, and the final abnormality detection and quality evaluation results are output by fusing statistical constraints. The present application has good detection effect and fast running speed. It is not only suitable for quality evaluation of periodic data, but also suitable for quality evaluation of non-stationary data. Through the above steps S1-S4, the abnormality detection and quality evaluation of the input time series data can be realized, which provides guarantee for subsequent time series prediction, time series analysis and the like.
[0163] In view of the poor performance of the existing deep learning-based time series data quality evaluation method on segment anomaly, the present application selects to introduce a variational model decomposition (VMD) to first perform data enhancement on the detection data. VMD can remove part of the larger noise through frequency domain decomposition, so as to better preserve the expected data distribution and pattern. Then, statistical constraints are introduced to assist the deep learning method in anomaly detection, and a correction mechanism is added to ensure good performance in the case of large-area data anomaly. By introducing statistical constraints and variational model decomposition, the detection accuracy on segment anomaly is greatly improved.
[0164] In view of the difficulty of setting an abnormal threshold for the existing deep learning-based time series data quality evaluation method, the present application selects to label a small sample data set. Abnormality detection is first performed on the data set, and an energy valley optimization algorithm (EVO) is introduced to adaptively find the abnormal threshold corresponding to the best abnormal detection result through continuous iteration, thereby realizing adaptive optimization of the threshold.
[0165] Embodiment 2
[0166] Please refer to Figure 2 , Figure 2 is a structural schematic diagram of a time series data anomaly detection and quality evaluation system disclosed by the embodiment of the present application, which can realize anomaly detection of time series data, and specifically comprises:
[0167] A data preprocessing module is configured to access high-quality time series data offline, and perform data preprocessing on the high-quality time series data to obtain clean training data.
[0168] a time series autoregressive module, configured to establish an initial time series data autoregressive model; input training data into the initial time series data autoregressive model, and perform back propagation offline update of network parameters of the initial time series data autoregressive model; after repeated multiple times, obtain a trained time series data autoregressive model;
[0169] an adaptive threshold calculation module, configured to perform adaptive optimization of a threshold using an EVO algorithm and the trained time series data autoregressive model on a labeled small sample data set, and obtain an optimal threshold;
[0170] an anomaly detection and evaluation module, configured to input time series data to be detected into the trained time series data autoregressive model, judge whether a mean square error of a calculation result of the trained time series data autoregressive model and the time series data to be detected exceeds the optimal threshold, and output a final anomaly detection and quality evaluation result by fusing statistical constraints.
[0171] In an optional embodiment, the time series data anomaly detection and quality evaluation method fusing an attention mechanism and statistical constraints comprises: a) offline access to high-quality time series data and pre-processing; b) establishing an initial time series data autoregressive model; training network parameters by back propagation using training data; after repeated multiple times, obtaining a trained time series data autoregressive model; c) performing adaptive optimization of a threshold using an EVO algorithm and the trained time series data autoregressive model on a labeled small sample data set, and obtaining an optimal threshold; d) inputting time series data to be detected into the trained time series data autoregressive model, judging whether a mean square error of a calculation result of the model and the time series data to be detected exceeds the optimal threshold, and outputting a final anomaly detection and quality evaluation result by fusing statistical constraints.
[0172] Embodiment 3
[0173] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of a time series data anomaly detection and quality evaluation device fusing an attention mechanism and statistical constraints disclosed by an embodiment of the present application. Wherein, Figure 3 The device described can be used for time series data quality evaluation, such as anomaly detection of time series data, etc., and the embodiments of the present application are not limited.
[0174] As Figure 3 shown, the device can include a processor and a memory, the memory storing computer instructions, the processor configured to execute the computer instructions stored in the memory, when the computer instructions are executed by the processor, the electronic device implements the steps of the method described in the above embodiments, and can achieve the same technical effects as the above method.
[0175] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, memory. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules typically perform the functions and / or methods described in the embodiments of the present invention.
[0176] The processor executes various functional applications and data processing by running programs stored in memory, such as the method provided in Embodiment 1 of the present invention.
[0177] Example 4
[0178] Embodiment 4 of the present invention also provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the method described in the above embodiments and achieves the same technical effect as the above method.
[0179] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0180] These computer readable program instructions can be provided to a computer of an apparatus, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the
[0181] The program code embodied on the computer readable media can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0182] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).
[0183] Of course, a storage medium storing computer executable instructions for performing operations according to any embodiment of the present application is not limited to the above method operations, and can also perform related operations in the method according to any embodiment of the present application.
[0184] The above specific embodiments describe the purpose, technical solutions and advantages of the present application in further detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints, characterized in that, include: High-quality time-series data is accessed offline, and data preprocessing is performed on the high-quality time-series data to obtain clean training data; Establish an initial time-series data autoregressive model; input the training data into the initial time-series data autoregressive model, and perform backpropagation on the initial time-series data autoregressive model to update the network parameters offline; repeat this process multiple times to obtain the trained time-series data autoregressive model; The optimal threshold is obtained by adaptively optimizing the threshold using the EVO algorithm and a trained time-series autoregressive model on a labeled small sample dataset. The time series data to be detected is input into a trained time series data autoregressive model. The mean square error between the calculation result of the trained time series data autoregressive model and the time series data to be detected is determined to exceed the optimal threshold. Statistical constraints are then integrated to output the final anomaly detection and quality assessment results.
2. The method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints according to claim 1, characterized in that, High-quality time-series data is accessed offline, and preprocessed to obtain clean training data, including: It can access high-quality multivariate time series data from multiple years offline and integrate and save all multivariate time series data into a CSV file; Read the CSV file, sort it by time and interpolate to fill in missing timestamps, remove outliers from the multivariate time series data, and fill in the missing values according to the average value; Variational model decomposition is used to break down the variable to be detected into high-frequency, mid-frequency and low-frequency components, thereby augmenting the original variable. All frequency domain components are concatenated with the remaining variables of the multivariate time series data column by column to construct clean training data.
3. The method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints according to claim 2, characterized in that, Variational model decomposition is used to break down the variable to be detected into high-frequency, mid-frequency, and low-frequency components, thereby augmenting the original variable, including: Constructing a variational constraint model: Assuming the detection variable f is decomposed into k components, and the given constraint is that the sum of all modes equals the original signal, the expression for the variational constraint model of VMD is as follows: Among them, u k ={u1,u2,…,u K } represents the modal functions; ω k ={ω1,ω2,…,ω K } represents the center frequency of each mode; The essence is about u k Perform Hilbert transform; Solving the variational problem: Define the Lagrangian function based on the objective function and constraints of the variational constrained model; perform a Fourier transform on all variables to transform them into the frequency domain; repeatedly update each mode and its center frequency alternately until convergence or the upper limit of iteration is reached, thus obtaining each u. k The time domain is then converted back using the inverse Fourier transform.
4. The method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints according to claim 1, characterized in that, The training data is input into the initial time series data autoregressive model, and the network parameters are updated offline by backpropagation of the initial time series data autoregressive model. After repeating this process multiple times, a trained time-series autoregressive model is obtained, including: Input clean training data x, and downsample it using average pooling (AvgPooling(·)) to obtain multi-scale time series data X = {x0,…,x}. M }, where M is the scale number, and it is projected into the first high-dimensional feature X = {x0,…,x} through the embedding layer Embed(·). M The process is as follows: X = AvgPooling(x) X = Embed(X) Multiple stacked sequence fusion modules are configured to perform information mixing on the first high-dimensional feature X and extract the underlying patterns of the high-dimensional data in the mixed first high-dimensional feature X; for the l-th sequence fusion module TSFM l (·), input is X l-1 Output X l It can be represented as: X l =TSFM l (X l-1 ) After passing through L stacked sequence fusion modules, the output of the sequence fusion module is: The pre-designed feature extractor Extractor(·) is used to extract deeper value information from the clean training data x and project it into a second high-dimensional feature. Will Input the scale into the predictor Prediction m (·), and with the second high-dimensional feature CrossAttention m The (·) operation, where m∈{0,…,m}, yields the output of the attention submodule. The formula for calculating cross attention is: Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively; d k The dimension of the key matrix is represented by T; T represents the transpose. The output of the attention submodule Input the scale into the Regression function. m In (·), the components reconstructed from the time series data autoregressive model are accumulated. Repeat the above steps, using MSE as the loss function for parameter optimization and weight updates; use the Adam optimizer and the OneCycleLR strategy to dynamically adjust the learning rate to achieve better optimization of the model parameters, and finally output the optimal network parameters. Import the optimal network parameters into the initial time series data autoregressive model to obtain the trained time series data autoregressive model.
5. The method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints according to claim 4, characterized in that, The l-th sequence fusion module TSFM l The workflow of (·) is as follows: In the l-th sequence fusion module, the first high-dimensional feature X is decomposed into multiple scales using SeriesDecomp(·). For each scale m∈{0,…,M}, the seasonal terms are decomposed. and trend items It can be represented as: in, These are the latent variables to be decomposed; This leads to the multi-scale seasonal terms. and trend items Trend terms at multiple scales Perform a TrendMix(·) operation to combine the trend term with the most refined seasonal term. Adding them together, we get: Therefore, the output of the l-th sequence fusion module is 6. The method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints according to claim 1, characterized in that, The optimal threshold is obtained by adaptively optimizing the threshold using the EVO algorithm and a trained time-series autoregressive model on a labeled small sample dataset, including: For the variables that need to be detected, a small sample dataset containing various anomalies is constructed and manually labeled. The constructed labeled small sample dataset is decomposed and concatenated using VMD and then input into a trained time series data autoregressive model to obtain the reconstructed components. The reconstructed components are summed column by column to obtain the reconstructed data of the detected variable, and the mean square error (MSE) of the reconstructed data of the detected variable and the original data of the detected variable is calculated. The lower and upper limits of the mean square error threshold are manually set, the threshold is initialized, and the result of anomaly detection is obtained by comparing it with the mean square error. The F1_Score is calculated by combining the result of anomaly detection with the manually labeled tag. An energy valley optimization algorithm is introduced, using F1_Score as the fitting function. Through continuous iteration, the threshold corresponding to the highest F1_Score is found, and this threshold can be regarded as the optimal threshold.
7. The method for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints according to claim 1, characterized in that, The time-series data to be detected is input into a trained time-series autoregressive model. The mean squared error between the calculated result of the trained time-series autoregressive model and the time-series data to be detected is determined to exceed an optimal threshold. Statistical constraints are then integrated to output the final anomaly detection and quality assessment results, including: Access the time series data to be detected and perform data preprocessing on the time series data to be detected; The preprocessed time series data is input into the trained time series data autoregressive model to regress the reconstructed components. All reconstructed components are added together to obtain the reconstructed data. The MSE of the reconstructed data and the original data of the detection variable is calculated to obtain the deviation between the two. The deviation is compared with the optimal threshold. If the deviation is greater than the threshold, it is an anomaly; otherwise, it is normal. Based on this judgment, the anomaly detection result is output. By introducing statistical constraints, boundary values are set and first-order difference judgments are performed on the preprocessed time series data to obtain anomaly detection results using statistical methods. By combining the anomaly detection results from deep learning and statistical methods, preliminary detection results are obtained. Further corrections are made based on these preliminary results, and the corrected values are compared with the original time-series data to obtain the final anomaly detection results. The final quality of the time-series data is evaluated based on the proportion of outliers detected in the time-series data.
8. A time-series data anomaly detection and quality assessment system integrating attention mechanisms and statistical constraints, characterized in that, include: The data preprocessing module is used to access high-quality time-series data offline and preprocess the high-quality time-series data to obtain clean training data. The time-series autoregressive module is used to establish an initial time-series data autoregressive model; the training data is input into the initial time-series data autoregressive model, and the network parameters are updated offline through backpropagation; after repeating this process multiple times, a trained time-series data autoregressive model is obtained. The adaptive threshold calculation module is used to adaptively optimize the threshold on a labeled small sample dataset using the EVO algorithm and a trained time-series autoregressive model to obtain the optimal threshold. The anomaly detection and evaluation module is used to input the time series data to be detected into a trained time series data autoregressive model, determine whether the mean square error between the calculation result of the trained time series data autoregressive model and the time series data to be detected exceeds the optimal threshold, and output the final anomaly detection and quality evaluation results by integrating statistical constraints.
9. A device for anomaly detection and quality assessment of time-series data that integrates attention mechanisms and statistical constraints, characterized in that, The device includes a processor and a memory, wherein the memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the time-series data anomaly detection and quality assessment method that integrates attention mechanisms and statistical constraints as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the time-series data anomaly detection and quality assessment method that integrates attention mechanisms and statistical constraints as described in any one of claims 1 to 7.