Deep learning-based highly unbalanced time series data abnormal value detection method

Abstract

The invention discloses a highly unbalanced time series data abnormal value detection method based on deep learning, and the method comprises the steps: S1, capturing the dynamic characteristics of a time series data set based on an abnormal value detection model of a time convolution network self-coding TCN-AE frame, and recognizing a potential abnormal value; s2, optimizing the data set based on adaptive synthetic sampling ADASYN so as to balance the abnormal value data set; s3, performing global optimization on TCN-AE abnormal value detection model parameters after ADASYN adaptive synthesis sampling based on a Raccoon longnose optimization algorithm COA, so as to obtain a minimum error in a low-abnormal-rate time sequence data set; and S4, evaluating the whole detection process through five evaluation indexes. According to the method, the abnormal value identification algorithm is optimized and improved, so that the potential abnormal value in the highly unbalanced data set can be identified more accurately, and the identification capability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data processing, and in particular to a method for detecting outliers in highly unbalanced time series data based on deep learning. Background Art

[0002] At present, the methods for time series anomaly detection in academia can be mainly divided into statistical methods, traditional machine learning methods and deep learning methods. Traditional outlier detection methods mainly use statistical methods. The main method is to establish a likelihood ratio test that obeys a normal or certain distribution statistic to determine whether there are outliers in the time series. However, it may not be effective for detecting some outliers such as sudden changes or long-term trend deviations. Another research method is not to focus on the outliers themselves, but to further observe the anomalies brought by the outliers to the model parameter estimation. An existing ARIMA-based outlier detection method has the basic idea of ​​approximating missing values ​​for outliers, fitting the ARIMA model with normal data in the time series, and judging whether the missing values ​​are abnormal based on their test statistics. In addition, there are some other statistical methods for outlier detection, such as the outlier detection method based on extreme value study deviation (ESD), the outlier detection method based on standardized Z-score, and the anomaly detection method based on distance and density (GMBR-DD).

[0003] Traditional machine learning is also one of the popular time series outlier detection technologies. This type of method is widely used in time series outlier identification tasks in different fields. For example, the proposed ARIMA-SVR model detects outliers in hydrological time series data affected by complex factors such as weather and terrain. This method uses SVR regression to predict the nonlinear part and then combines ARIMA to predict the linear autocorrelation part of the time series, and generates a confidence interval with a specific confidence level. If the actual observation value is not within the interval, it is judged as an outlier. Using the real data of the Liuhe measurement station in the Chuhe River Basin, it is found that the ARIMA-SVR model can still accurately detect outliers in the time series while maintaining high specificity and sensitivity. Others add a risk function to the structure of the SVM algorithm to obtain better time series smoothness, and then combine the phase space reconstruction theory to propose an outlier detection method based on regression judgment. The simulation experiments were carried out on two sets of data, Henon chaotic time series and NAR process. The results show that most dependent variable outliers can be detected by this method.

[0004] The structure of the time series outlier detection model based on deep learning is more complex, and is often proposed in the form of a hybrid model that integrates data preprocessing, parameter optimization algorithm, and autoencoder framework. For example, there is a proposed deep learning supervised time series anomaly detection method Multi-headCNN-RNN that combines CNN and RNN models. This method innovatively proposes to use multiple independent CNN convolution modules to extract outliers from multiple sensors, and avoids the problem of insufficient learning of data anomalies by processing the abnormal data of each sensor separately. The RNN model is then used to predict the time series data, so as to compare and find outliers. After that, it was evaluated based on the actual industrial case data provided by the author's industrial partners, and it was found that the Multi-headCNN-RNN model performed well on point anomaly, global anomaly, and local anomaly data sets. In the prior art, a new autoencoder time series anomaly detection model (DA-CBG-AE) is proposed for time series data with weak periodicity and high complexity. The model integrates a new sliding window to set the sliding window size for the time series periodically, uses a convolutional neural network to extract high-dimensional spatial features of the time series, and proposes a stacked Dropout bidirectional gate recurrent unit network as the basic structure of the autoencoder. Finally, a double-layer attention mechanism is introduced to further extract features, so that more critical time series can be selected to improve the accuracy of anomaly detection. Finally, by comparing DA-CBG-AE with 6 benchmark models on 8 data sets, it obtains a higher F1 value. Others integrate an automatic hyperparameter optimization mechanism, which generates optimized model instances with the best architecture and training parameters from the model pool of five selected models (i.e., temporal convolutional network (TCN), InceptionTime, MiniRocket, residual network (ResNet) and LSTM). The time series generative adversarial network (TimeGAN) is used as a synthetic data set generator to test the model effect, as shown in Table 1 below.

[0005] Table 1. Related literature on time series outlier detection

[0006]

[0007]

[0008] In summary, existing outlier detection models cannot learn the rules of outliers in time series data with few outlier samples and low anomaly rates, cannot meet the needs of outlier identification in complex time series, and are even more difficult to effectively identify highly unbalanced and low anomaly rate data sets to achieve accurate predictions. Summary of the invention

[0009] The purpose of the present invention is to provide a method for detecting outliers in highly unbalanced time series data based on deep learning. Based on the current research foundation of deep learning methods, outlier detection is performed on time series data with low anomaly rates and the results are analyzed. The minimum error can be obtained in the time series data set with low anomaly rates to solve the shortcomings of the prior art.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] The outlier detection method for highly unbalanced time series data based on deep learning includes the following steps:

[0012] S1: The outlier detection model based on the temporal convolutional network autoencoder (TCN-AE) framework captures the dynamic characteristics of time series datasets and identifies potential outliers.

[0013] S2: Optimize the dataset based on adaptive synthetic sampling ADASYN to balance the outlier dataset;

[0014] S3: Global optimization of the TCN-AE outlier detection model parameters after ADASYN adaptive synthetic sampling is performed based on the coati optimization algorithm COA to obtain the minimum error in low anomaly rate time series data sets;

[0015] S4: The entire detection process is evaluated using five evaluation indicators: accuracy, precision, recall, the harmonic mean of precision and recall F1-score, and AUC-ROC.

[0016] Furthermore, in the TCN layer of the temporal convolutional network autoencoder TCN-AE framework, causal convolution is used to process the input time series. The formula of causal convolution is:

[0017]

[0018] Among them, y t is the value of the output sequence at time step t, x t-k is the value of the input sequence at time step tk, W k is the weight of the convolution kernel, K is the length of the convolution kernel, and f is the activation function, ReLU or other nonlinear activation functions are selected;

[0019] In order to expand the receptive field, dilated convolution is introduced in the TCN layer. The formula of dilated convolution is:

[0020]

[0021] Among them, r is the expansion factor, which determines the interval between elements in the convolution kernel. Residual connections are also used in the TCN layer to improve the propagation of information in the network. The output can be expressed as:

[0022]

[0023] By combining causal convolution, dilated convolution, and residual connection, TCN can effectively capture long-distance dependencies while maintaining low computational complexity.

[0024] Furthermore, in the encoder and decoder of the temporal convolutional network autoencoder TCN-AE framework, the data is first positionally encoded by a combination of a set of sine and cosine functions to help the model understand the relative positions of elements. The mathematical formula for position encoding is as follows:

[0025]

[0026]

[0027] Among them, pos is the position of the element in the sequence, i is the index of the position encoding dimension, and d is the total dimension of the position encoding;

[0028] The position encoding is added to the embedded input features. The input sequence is then embedded in a multi-dimensional space, and the relationship between elements in the sequence is captured through a multi-head self-attention mechanism. In the self-attention mechanism, the encoding vector e(t) of each position is used to calculate the attention distribution, and then the output of each position is obtained by weighted averaging the encodings of other positions.

[0029] The formula for calculating the self-attention weight is:

[0030]

[0031] The formula for multi-head self-attention calculation is:

[0032] MultiHead(e(t))=Concat(head1,head2,…,head h )W O ,

[0033] in

[0034] head1=Attention(e(t)W i Q ,e(t)W i K ,e(t)W i V ),

[0035] head i represents the output of the ith attention head; in a multi-head self-attention mechanism, there are usually multiple such heads, each of which can learn different attention weights; Attention(QWi Q ,KW i K ,VW i V ) is an attention function that uses the query (Q), key (K), and value (V) to compute the attention output.

[0036] Furthermore, the optimization method in S2 is as follows:

[0037] For each minority class sample x i , first calculate its k nearest neighbor samples; then evaluate the proportion of majority class samples in its neighbors. If the proportion is high, it is considered that the samples around the sample are more difficult to classify; ADASYN dynamically determines the number of new samples to generate based on the difficulty:

[0038]

[0039] Where K is the sample x i By increasing the number of neighbors, ADASYN can generate more samples in areas with fewer samples, thereby enhancing the model's ability to recognize minority classes.

[0040] Furthermore, the first stage of the coati optimization algorithm COA in S3 is to simulate the strategy of the coati population when attacking lizards for modeling: the position of the best individual in the current population is assumed to be position X, and the model assumes that half of the coatis will climb up the tree and the other half will wait for the lizard to land. The position of the coatis on the tree is simulated by a mathematical formula as follows:

[0041]

[0042] After the lizard lands, it is placed at a random position in the search space, and based on this random position, the raccoon on the ground moves in the search space to update its position:

[0043]

[0044]

[0045] Among them, N is the population size, t is the current iteration number, lb j represents the lower bound of the j-th dimension variable, ub j represents the upper bound of the j-th dimension variable, fitness() is used to calculate the fitness; r represents a random number between [0,1], and I represents a random integer from the integer set {1,2}; therefore, represents the new position of the lizard after it falls to the ground. This represents the j-th dimension variable value of the i-th individual in the current iteration, that is, the first half of the population individuals climb trees to update their positions, and the second half of the population individuals wait on the ground for the lizard to land and update their positions; if the new position calculated for each coati improves the value of the objective function, the new position is acceptable, otherwise, the coati remains in the previous position and performs a greedy selection:

[0046]

[0047] Furthermore, the second phase of the coati optimization algorithm COA in S3, which updates the position of the coati in the search space, is mathematically modeled based on the coati's natural behavior when escaping from a predator: when a predator attacks a coati, the coati flees its position, and the coati's actions in this strategy put it in a safe position close to its current position. In simulating this behavior, a random position is generated near the position of each coati:

[0048]

[0049]

[0050] Among them, r represents a random number between [0,1], T is the maximum number of iterations, it can be seen that Are the upper and lower bounds of the j-th dimension variable as the number of iterations increases. If the newly calculated position improves the value of the objective function, the position is acceptable and simulated, that is, another greedy selection is performed.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The outlier detection method for highly unbalanced time series data based on deep learning of the present invention can more accurately identify potential outliers in highly unbalanced data sets and improve the recognition ability of the model by optimizing and improving the outlier recognition algorithm. When extended to practical applications, it can help banks and financial institutions better detect potential fraudulent behavior or abnormal transactions, or help doctors identify abnormalities in patient data, thereby improving the accuracy of diagnosis. In the manufacturing industry, the optimized outlier recognition algorithm can help detect equipment failures or production line abnormalities, improve production efficiency and product quality. Overall, by optimizing and improving the outlier recognition algorithm, the accuracy and reliability of data analysis can be improved in various fields, helping decision makers make more accurate judgments and predictions. The promotion and application of the present invention will bring more possibilities to all walks of life, accelerate innovation, improve efficiency, and reduce risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a simulation diagram of the first stage of predation by the coati of the present invention;

[0054] Figure 2 This is a simulation diagram of the second stage of the coati escaping from pursuit according to the present invention;

[0055] Figure 3 It is a flow chart of the detection method of the present invention;

[0056] Figure 4 It is the AUC-ROC curve diagram of different abnormal rate data sets of the present invention;

[0057] Figure 5 AUC-ROC curve diagram before and after optimization of the model of the present invention;

[0058] Figure 6 This is the recognition effect diagram before and after optimization of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 3 The method for detecting outliers in highly unbalanced time series data based on deep learning provided by an embodiment of the present invention comprises the following steps:

[0061] S1: The outlier detection model based on the temporal convolutional network autoencoder TCN-AE framework captures the dynamic characteristics of time series data sets and identifies potential outliers. The TCN layer of the TCN-AE framework uses causal convolution and residual connection to efficiently capture the long-term and short-term dependencies in time series data. The encoder part of the model uses the Transformer encoder and decoder. Compared with the traditional recurrent neural network, the encoder-decoder is more computationally efficient and can process data in parallel, avoiding the gradient vanishing problem. In the overall autoencoder framework, TCN is responsible for extracting the potential temporal feature representation from the input time series, while the encoder-decoder tries to learn the long-term regular representation of the data based on the potential temporal features, thereby realizing the detection of outliers.

[0062] Specifically, in the TCN layer, causal convolution is used to process the input sequence to ensure that the output of the current time step only depends on the current and previous inputs, thereby maintaining the temporal order. The formula for causal convolution is:

[0063]

[0064] Among them, y t is the value of the output sequence at time step t, x t-k is the value of the input sequence at time step tk, W k is the weight of the convolution kernel, K is the length of the convolution kernel, and f is the activation function, usually ReLU or other nonlinear activation functions. In order to expand the receptive field, TCN introduced dilated convolution, which introduces spacing between convolution kernel elements and can capture longer context information with less computing resources. The formula for dilated convolution is:

[0065]

[0066] Here, r is the expansion factor, which determines the spacing between elements in the convolution kernel; TCN also uses residual connections to improve the propagation of information in the network. The output can be expressed as:

[0067]

[0068] This structure makes the output of each layer not only depend on the calculation results of the current layer, but also directly receive input, making the network easier to optimize during training. By combining causal convolution, dilated convolution and residual connection, TCN can effectively capture long-distance dependencies while maintaining low computational complexity; due to its superior performance and flexibility, TCN has been widely used in many application fields such as time series prediction, speech signal processing and natural language processing.

[0069] In Transformer, data is first positionally encoded by a combination of sine and cosine functions to help the model understand the relative positions of elements; position encoding is usually composed of a combination of sine and cosine functions, whose frequencies and offsets are adjusted according to the position. These encodings have different values ​​at each position to ensure that the encoding at each position is unique. The mathematical formula for position encoding is as follows:

[0070]

[0071]

[0072] Among them, pos is the position of the element in the sequence, i is the index of the position encoding dimension, and d is the total dimension of the position encoding. These encodings are added to the embedded input features to fuse the position information.

[0073] The input sequence is then embedded into a multi-dimensional space, and the relationship between elements in the sequence is captured through a multi-head self-attention mechanism, which allows the model to establish complex dependencies between different positions without a fixed local window. In the self-attention mechanism, the encoding vector e(t) of each position is used to calculate the attention distribution, and then the output of each position is obtained by weighted averaging the encodings of other positions, thereby better capturing patterns and trends in the sequence.

[0074] The formula for calculating the self-attention weight is:

[0075]

[0076] The formula for multi-head self-attention calculation is:

[0077] MultiHead(e(t))=Concat(head1,head2,…,head h )W O ,

[0078] in

[0079] head1=Attention(e(t)W i Q ,e(t)W i K ,e(t)W i V ),

[0080] head i represents the output of the ith attention head; in a multi-head self-attention mechanism, there are usually multiple such heads, each of which can learn different attention weights; Attention(QW i Q ,KW i K ,VW i V ) is an attention function that uses the query (Q), key (K), and value (V) to compute the attention output. The specific form of this function is usually based on dot-product attention (Dot-ProductAttention) or other variants such as weighted average or scaled dot-product attention, QW i Q , KW i K and VW i V are linear transformations for query, key, and value, respectively, which are weight matrices W i Q , W i K and W iV Matrix multiplication with the input matrices Q, K, and V. These linear transformations are performed to map the input into a space suitable for attention computation.

[0081] In summary, the outlier detection framework based on the TCN-AE autoencoder framework in S1 effectively combines the advantages of temporal convolutional networks and autoencoders, and can efficiently capture the dynamic features in time series data. Compared with traditional outlier detection methods, this model has significant detection accuracy when processing high-frequency data and complex patterns. Future research can further explore the performance of the TCN-AE model in different application scenarios, as well as its combination with other deep learning techniques to enhance the ability of outlier detection.

[0082] S2: Adaptive synthetic sampling ADASYN is used to optimize the dataset to balance the outlier dataset. Specifically, in the field of machine learning and data mining, the prediction performance of the model is often significantly affected when faced with highly imbalanced datasets. Due to the scarcity of minority class samples, the model may be biased towards the majority class, resulting in poor recognition of the minority class. To overcome this problem, resampling techniques are widely used to balance the number of samples of each class. The main resampling methods include oversampling and undersampling, resampling, and adaptive synthetic sampling. Adaptive synthetic sampling (ADASYN) is another improved oversampling technique that aims to solve the imbalance problem by generating new minority class samples. Similar to SMOTE, ADASYN is also based on the idea of ​​generating new samples, but it considers the difficult samples around each minority class sample when generating samples.

[0083] ADASYN works as follows: For each minority class sample x i , first calculate its k nearest neighbor samples. Then, evaluate the proportion of majority class samples in its neighbors. If the proportion is high, it is considered that the samples around the sample are more difficult to classify. ADASYN will dynamically determine the number of new samples to generate based on the difficulty:

[0084]

[0085] Where K is the sample x i In this way, ADASYN can generate more samples in areas with fewer samples, thereby enhancing the model's recognition ability for minority classes.

[0086] S3: Global optimization of the TCN-AE outlier detection model parameters after ADASYN adaptive synthetic sampling based on the coati optimization algorithm COA is used to obtain the minimum error in low anomaly rate time series data sets; In this step, the coati optimization algorithm (COA) is used to find the adaptive resampling parameter k and the hyperparameters in the TCN-AE autoencoder framework. COA simulates two natural behaviors of coatis in nature when hunting iguanas (exploration phase) and escaping from predators (escape phase). The COA algorithm simulates the implementation steps of COA from the two stages of exploration and exploitation, and mathematically models it. The results of COA are compared with the results of 11 well-known metaheuristic algorithms. The simulation results show that COA has obvious advantages and strong competitiveness.

[0087] The first stage of COA is to simulate the strategy of the raccoon population when attacking lizards: the position of the best individual in the current population is assumed to be position X. The model assumes that half of the raccoons will climb the tree and the other half will wait for the lizard to land. Therefore, the following formula is used to mathematically simulate the position of the long-nosed coatis on the tree:

[0088]

[0089] After the lizard lands, it is placed at a random position in the search space, and based on this random position, the raccoon on the ground moves in the search space to update its position:

[0090]

[0091]

[0092] Among them, N is the population size, t is the current iteration number, lb j represents the lower bound of the j-th dimension variable, ub j represents the upper bound of the j-th dimension variable, fitness() is used to calculate the fitness, r represents a random number between [0,1], and I represents a random integer from the integer set {1,2}. Therefore, Indicates the new position of the lizard after it falls to the ground; This represents the j-th dimension variable value of the i-th individual in the current iteration. In short, the first half of the individuals in the population climb trees to update their positions, and the second half of the individuals in the population wait on the ground for the lizard to land and update their positions.

[0093] If the new position calculated for each coati improves the value of the objective function, the new position is accepted, otherwise the coati remains in its previous position, that is, a greedy selection is performed:

[0094]

[0095] Specifically, the first stage has two processes as follows: Figure 1 shown.

[0096] The second phase of updating the coati’s position in the search space is mathematically modeled based on the coati’s natural behavior when fleeing from a predator; when a predator attacks a coati, the coati flees its position. The coati’s actions in this strategy put it in a safe position close to its current position. To simulate this behavior, a random position is generated near each coati’s position.

[0097]

[0098]

[0099] Among them, r represents a random number between [0,1], T is the maximum number of iterations, it can be seen that is the upper and lower bounds of the j-th dimension variable updated with the number of iterations. If the newly calculated position improves the value of the objective function, the position is acceptable and simulated, that is, another greedy selection is performed. The process of the second stage is as follows Figure 2 shown.

[0100] S4: The whole detection process is evaluated by five evaluation indicators: accuracy, precision, recall, harmonic mean of precision and recall F1-score, and AUC-ROC; specifically, there are many commonly used evaluation criteria in outlier detection tasks, and each indicator has its own emphasis and limitations. The present invention combines multiple indicators to comprehensively evaluate the performance of the model, especially when facing a class-imbalanced data set, a single indicator may mislead the performance of the model, specifically:

[0101] Accuracy is one of the most basic evaluation indicators in the classification model. This indicator provides a basic measure of the overall performance of the model. It is a more reliable evaluation standard when the category distribution is relatively balanced. It is used to measure the proportion of correct predictions of the model in all samples. In the task of outlier detection, the formula for accuracy is:

[0102]

[0103] Among them, TP (True Positive) represents the anomaly points correctly detected by the model, TN (True Negative) represents the normal points correctly detected by the model, FP (False Positive) represents the normal points incorrectly marked as anomalies by the model, and FN (False Negative) represents the anomaly points incorrectly marked as normal by the model. However, in the anomaly detection problem, there are often very few anomalies and the data set is strongly unbalanced. At this time, although the model may have a high accuracy, this performance may be just because the model is predicting normal points most of the time and ignoring anomalies. Therefore, although accuracy is a simple and easy-to-use indicator, it cannot fully reflect the effectiveness of the model in an imbalanced data set.

[0104] Precision is a measure of how many of the positive classes (outliers) in the model's prediction results are truly positive. False positives in anomaly detection often come at a cost, because false positives can cause unnecessary intervention and waste of resources. Therefore, precision is very important, especially in scenarios where false positives have a greater impact on the system. For example, in fraud detection or fault diagnosis, if the system frequently reports false positives, it may affect user experience or increase operating costs. For outlier detection tasks, the calculation formula for precision is as follows:

[0105]

[0106] The precision reflects the reliability of the model in detecting anomalies. A high precision means that the model produces fewer false positives, that is, most of the points detected as anomalies are true anomalies.

[0107] Recall is a measure of the proportion of all positive classes (outliers) that the model can detect. In anomaly detection, recall reflects the sensitivity of the model in detecting outliers, that is, how many actual outliers it can identify. In some applications, the cost of false negatives may be higher than false positives. For example, in medical diagnosis, missing a disease may have serious consequences, so it is necessary to maximize the recall rate, even if this may lead to a decrease in precision. The definition of recall is:

[0108]

[0109] In outlier detection, recall is particularly important and is used to evaluate the detection ability of the model, especially when outliers are rare. A higher recall means that the model can detect more outliers and reduce missed reports.

[0110] F1-score is the harmonic mean of precision and recall, aiming to achieve a balance between the two. In anomaly detection tasks, F1-score can effectively evaluate the overall performance of the model, especially when the number of anomalies is small. When precision and recall conflict, using a single indicator may not fully reflect the true ability of the model. In this case, F1-score becomes particularly important. Its formula is:

[0111]

[0112] F1-score combines the advantages of precision and recall, and is particularly useful in the case of data imbalance. By calculating the harmonic mean, F1-score provides a comprehensive evaluation of the two indicators, which can find a suitable compromise between precision and recall. The value of F1-score ranges from 0 to 1. The higher the value, the better the model performance.

[0113] AUC-ROC is an important indicator for evaluating the classification performance of the model at different thresholds. It is particularly suitable for model evaluation in binary classification problems. The ROC curve (Receiver Operating Characteristic Curve) is a curve that plots the relationship between the true positive rate (True Positive Rate, TPR) and the false positive rate (False Positive Rate, FPR). AUC (Area Under the Curve) represents the area under the ROC curve, and the value is between 0.5 and 1. The closer the AUC is to 1, the better the model performance is. The calculation formulas for TPR and FPR are:

[0114]

[0115]

[0116] AUC-ROC measures the overall performance of the model at different classification thresholds. It not only evaluates the classification ability of the model, but also reveals the trade-off relationship at different thresholds. For example, when the AUC value is high, it means that the model performs well in distinguishing positive and negative categories. For anomaly detection tasks, AUC-ROC is a very important indicator, especially when the data set is highly unbalanced. AUC can comprehensively measure the pros and cons of the model.

[0117] In order to further better explain the embodiment of the present invention, the following specific implementation case is also provided: Using the data set in Table 2:

[0118] Table 2 Outlier detection data

[0119]

[0120] The performance of outlier recognition of the above data based on the TCN-AE outlier autoencoder framework is as follows:

[0121] The study found that there are also significant differences in the detection effects of outlier detection models on data sets with different anomaly rates, such as Figure 4 Shown: From Figure 4 It can be clearly found that the area of ​​the lower right area of ​​the AUC-ROC curve of data set 1 is as follows Figure 4 (a) is significantly lower than that of dataset 2. Figure 4 (b) with Dataset 3 Figure 4 (c) This is because the anomaly rate of dataset 1 is 0.524%, which is much lower than the anomaly rate of dataset 2 (2.16%) and dataset 3 (1.35%). The detection effect of datasets with higher anomaly rates is obviously better than that of datasets with lower anomaly rates. Dataset 1 is a highly unbalanced outlier dataset, in which the proportion of outliers only accounts for a small part of the total value. This makes the detection model unable to fully learn the laws and characteristics of the outliers in the dataset, thereby reducing the accuracy of outlier identification.

[0122] Outlier recognition performance of ADASYN-COA-TCN-AE after optimization:

[0123] Based on the COA optimization algorithm, the TCN-AE outlier detection model after ADASYN adaptive synthetic sampling is globally optimized, and the hyperparameter values ​​of the TCN-AE model are shown in Table 3:

[0124] Table 3 TCN-AE algorithm hyperparameter meaning and value

[0125]

[0126] The TCN-AE algorithm optimized by ADASYN adaptive synthetic sampling and COA optimization algorithm is trained and tested on the test set and training set of the highly unbalanced outlier dataset 1, and the training effect is compared with that of the TCN-AE algorithm before optimization, as shown in Table 4:

[0127] Table 4 Comparison of training effects of TCN-AE algorithm before and after optimization algorithm

[0128]

[0129] As shown in Table 4, the accuracy of the optimized random forest model increased from 0.9177 to 0.9983, which is closer to 1; the recall rate increased from 0.5951 to 0.8724, and the F-score increased from 0.594 to 0.898. The improvement of the above indicators shows that the output effect of the TCN-AE algorithm has been improved, and its RMSE (root mean square error) has decreased from 0.0609 to 0.0313, making the algorithm more accurate in identifying outliers. Comparing the performance of the AUC-ROC curve of the model before and after optimization on the highly unbalanced outlier data set 1, as shown in Figure 5 Shown: From Figure 5 It can be clearly observed that the optimized AUC-ROC curve almost completely forms a right angle protruding to the upper left, see Figure 5 (b) The area on the lower right side of the curve is larger than before optimization, and the AUC is also increased from 0.90 before optimization to 0.99. Figure 6 , indicating that the optimization effect of the model is significant.

[0130] Since the model after adaptive sampling can fully learn the characteristics of outliers and normal values ​​in the data, the prediction effect will be improved. At the same time, thanks to the parameter optimization model for the overall TCN-AE autoencoder framework in terms of prediction accuracy and detection efficiency, the final outlier detection effect has been significantly improved. Figure 6 By observing the recognition effect of the model on Dataset 1 before and after optimization, it is found that the model before optimization is not effective in identifying some non-extreme outliers, which will cause certain omissions and the overall recognition rate is relatively low. The optimized model can identify outliers more accurately when the data produces non-extreme anomalies, thereby also bringing a corresponding improvement in the recognition rate of overall data outliers.

[0131] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting outliers in highly unbalanced time series data based on deep learning, characterized in that: The following steps are involved: S1: The outlier detection model based on the temporal convolutional network autoencoder (TCN-AE) framework captures the dynamic characteristics of time series datasets and identifies potential outliers. S2: Optimize the dataset based on adaptive synthetic sampling ADASYN to balance the outlier dataset; S3: Global optimization of the TCN-AE outlier detection model parameters after ADASYN adaptive synthetic sampling is performed based on the coati optimization algorithm COA to obtain the minimum error in low anomaly rate time series data sets; S4: The entire detection process is evaluated using five evaluation indicators: accuracy, precision, recall, the harmonic mean of precision and recall F1-score, and AUC-ROC.

2. The method for detecting outliers in highly unbalanced time series data based on deep learning according to claim 1, characterized in that: In the TCN layer of the temporal convolutional network autoencoder TCN-AE framework, causal convolution is used to process the input time series. The formula of causal convolution is: Among them, y t is the value of the output sequence at time step t, x t-k is the value of the input sequence at time step tk, W k is the weight of the convolution kernel, K is the length of the convolution kernel, and f is the activation function, ReLU or other nonlinear activation functions are selected; In order to expand the receptive field, dilated convolution is introduced in the TCN layer. The formula of dilated convolution is: Among them, r is the expansion factor, which determines the interval between elements in the convolution kernel. Residual connections are also used in the TCN layer to improve the propagation of information in the network. The output can be expressed as: By combining causal convolution, dilated convolution, and residual connection, TCN can effectively capture long-distance dependencies while maintaining low computational complexity.

3. The method for detecting outliers in highly unbalanced time series data based on deep learning as claimed in claim 2, characterized in that: In the encoder and decoder of the temporal convolutional network autoencoder TCN-AE framework, the data is first positionally encoded by a combination of a set of sine and cosine functions to help the model understand the relative positions of elements. The mathematical formula for position encoding is as follows: Among them, pos is the position of the element in the sequence, i is the index of the position encoding dimension, and d is the total dimension of the position encoding; The position encoding is added to the embedded input features. The input sequence is then embedded in a multi-dimensional space, and the relationship between elements in the sequence is captured through a multi-head self-attention mechanism. In the self-attention mechanism, the encoding vector e(t) of each position is used to calculate the attention distribution, and then the output of each position is obtained by weighted averaging the encodings of other positions. The formula for calculating the self-attention weight is: The formula for multi-head self-attention calculation is: MultiHead(e(t))=Concat(head1,head2,…,head h )W O , in head1=Attention(e(t)W i Q ,e(t)W i K ,e(t)W i V ), head i represents the output of the ith attention head; in a multi-head self-attention mechanism, there are usually multiple such heads, each of which can learn different attention weights; Attention(QW i Q ,KW i K ,VW i V ) is an attention function that uses the query (Q), key (K), and value (V) to compute the attention output.

4. The method for detecting outliers in highly unbalanced time series data based on deep learning according to claim 1, characterized in that: The optimization method in S2 is as follows: For each minority class sample x i , first calculate its k nearest neighbor samples; then evaluate the proportion of majority class samples in its neighbors. If the proportion is high, it is considered that the samples around the sample are more difficult to classify; ADASYN dynamically determines the number of new samples to generate based on the difficulty: Where K is the sample x i By increasing the number of neighbors, ADASYN can generate more samples in areas with fewer samples, thereby enhancing the model's ability to recognize minority classes.

5. The method for detecting outliers in highly unbalanced time series data based on deep learning according to claim 1, characterized in that: The first stage of the coati optimization algorithm (COA) in S3 is to simulate the strategy of the raccoon population when attacking lizards. The position of the best individual in the current population is assumed to be position X. The model assumes that half of the raccoons will climb the tree, and the other half will wait for the lizard to land. The mathematical formula is used to simulate the position of the coati on the tree as follows: After the lizard lands, it is placed at a random position in the search space, and based on this random position, the raccoon on the ground moves in the search space to update its position: Among them, N is the population size, t is the current iteration number, lb j represents the lower bound of the j-th dimension variable, ub j represents the upper bound of the j-th dimension variable, fitness() is used to calculate the fitness; r represents a random number between [0,1], and I represents a random integer from the integer set {1,2}; therefore, represents the new position of the lizard after it falls to the ground. This represents the j-th dimension variable value of the i-th individual in the current iteration, that is, the first half of the population individuals climb trees to update their positions, and the second half of the population individuals wait on the ground for the lizard to land and update their positions; if the new position calculated for each coati improves the value of the objective function, the new position is acceptable, otherwise, the coati remains in the previous position and performs a greedy selection:

6. The method for detecting outliers in highly unbalanced time series data based on deep learning according to claim 5, characterized in that: The second phase of the coati optimization algorithm COA in S3 updates the position of the coati in the search space by mathematically modeling the coati's natural behavior when fleeing from a predator: when a predator attacks the coati, the coati flees its position. The coati's actions in this strategy put it in a safe position close to its current position. When simulating this behavior, a random position is generated near each coati's position: Among them, r represents a random number between [0,1], T is the maximum number of iterations, it can be seen that Are the upper and lower bounds of the j-th dimension variable as the number of iterations increases. If the newly calculated position improves the value of the objective function, the position is acceptable and simulated, that is, another greedy selection is performed.

Images