Time sequence anomaly detection method based on self-supervised contrast learning

Abstract

The invention relates to a time series anomaly detection method based on self-supervised contrast learning. The method comprises the steps of obtaining time series data to be detected; to-be-detected time series data are input into the time series anomaly detection model for feature extraction and anomaly judgment, the anomaly probability or anomaly label of each time window is output, the input time series anomaly detection model comprises a feature extraction network based on A-SCINet and a classifier, and the classifier is used for classifying the anomaly probability or anomaly label of each time window. And pre-training the A-SCINet-based feature extraction network through a comparative learning self-supervised learning method, and training the pre-trained A-SCINet-based feature extraction network and the classifier through a pseudo tag. According to the method, on the premise that a large amount of labeled data is not needed, the detection precision and robustness of a complex abnormal mode are effectively improved, and efficient abnormal recognition of multi-dimension and multi-type time series data is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and machine learning, in particular to a time series anomaly detection method based on self-supervised contrastive learning. BACKGROUND

[0002] At present, in the field of time series anomaly detection, traditional methods rely on a large amount of labeled data and are strongly dependent on abnormal types, and when dealing with multi-dimensional and complex time series features, there are often problems of low detection accuracy and poor robustness. At the same time, existing anomaly detection methods based on supervised learning are difficult to effectively train in the case of scarce abnormal samples and fuzzy abnormal boundaries. In addition, many methods fail to fully utilize the potential information of unlabeled data, resulting in insufficient model generalization ability and difficulty in adapting to the changing abnormal patterns in actual industrial and Internet of Things environments. SUMMARY

[0003] The purpose of the present application is to provide a time series anomaly detection method based on self-supervised contrastive learning, which can effectively improve the detection accuracy and robustness of complex abnormal patterns without the need for a large amount of labeled data, and realize efficient anomaly recognition of multi-dimensional and multi-type time series data.

[0004] To achieve the above purpose, the present application provides the following scheme:

[0005] A time series anomaly detection method based on self-supervised contrastive learning, comprising:

[0006] Obtaining time series data to be detected;

[0007] Inputting the time series data to be detected into a time series anomaly detection model for feature extraction and anomaly judgment, and outputting the anomaly probability or anomaly label of each time window, wherein the input time series anomaly detection model comprises a feature extraction network based on A-SCINet and a classifier, the A-SCINet is a SCINet model with an embedded attention mechanism, the feature extraction network based on A-SCINet is pre-trained by a contrastive learning self-supervised learning method, and the feature extraction network based on A-SCINet after pre-training and the classifier are trained by pseudo labels.

[0008] Optionally, training the feature extraction network based on A-SCINet by a contrastive learning self-supervised learning method comprises:

[0009] Obtaining an original time series;

[0010] Preprocessing the original time series and dividing it into fixed-length time windows to obtain time segments;

[0011] selecting the time segment and injecting different types of abnormal disturbances to construct positive and negative samples;

[0012] extracting features of the positive and negative samples through the A-SCINet, inputting the extracted features into an MLP, and outputting a feature vector;

[0013] optimizing the feature vector through a contrast loss function.

[0014] Optionally, the different types of abnormal disturbances include global abnormality, trend abnormality, periodic abnormality, context abnormality, and shape abnormality.

[0015] Optionally, the feature extraction of the positive and negative samples through the A-SCINet includes:

[0016] inputting a standardized time series segment in the positive and negative samples into a sequence decomposition unit, performing smoothing processing on the standardized time series segment to obtain a trend component and a residual component;

[0017] processing the trend component through a linear layer to obtain a trend feature;

[0018] inputting the residual component into an SCI-Block unit of an SCI-Net model to perform recursive feature extraction, combining the recursive feature and the trend feature to obtain a final time series feature representation, wherein each SCI-Block unit is embedded with a dual self-attention network based on A 2 -Nets for dynamic weighting in feature extraction.

[0019] Optionally, the training of the pre-trained A-SCINet-based feature extraction network and the classifier through pseudo labels includes:

[0020] extracting features of the positive and negative samples through the pre-trained A-SCINet-based feature extraction network to obtain a feature vector;

[0021] According to the feature vector, obtaining a nearest neighbor set and a farthest neighbor set from the positive and negative samples;

[0022] dynamically generating pseudo labels according to the nearest neighbor set;

[0023] training the pre-trained A-SCINet-based feature extraction network and the classifier through a joint loss function, wherein the joint loss function includes a consistency loss calculated according to the nearest neighbor set, an inconsistency loss calculated according to the farthest neighbor set, and a cross-entropy loss calculated according to the pseudo labels.

[0024] Optionally, the acquiring the nearest neighbor set and the farthest neighbor set according to the feature vector comprises: according to the feature vector, selecting the nearest neighbor and the farthest neighbor sample in the feature space according to the K nearest neighbor principle, and acquiring the nearest neighbor set and the farthest neighbor set.

[0025] Optionally, the dynamically generating the pseudo label according to the nearest neighbor set comprises: dynamically generating the pseudo label by majority voting according to the known label or the historical pseudo label in the nearest neighbor set.

[0026] Optionally, the outputting the anomaly probability or the anomaly label of each time window comprises:

[0027] The output result of the A-SCINet-based feature extraction network is subjected to Softmax normalization processing to generate a probability distribution of each time window belonging to a normal or abnormal category, and an anomaly score is calculated according to the probability distribution to determine the abnormality degree of each time window.

[0028] The whole anomaly discrimination process of the present application not only guarantees efficient adaptive learning of unlabeled time series, but also improves the practicability and stability of the model in complex environments through fine probability scoring. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0030] Figure 1 A flow framework diagram of a time series anomaly detection method based on self-supervised contrastive learning according to an embodiment of the present application;

[0031] Figure 2 An A-SCINet architecture diagram according to an embodiment of the present application, wherein (a) is an SCI-Block unit diagram, and (b) is an A-SCINet architecture. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] This embodiment provides a time series anomaly detection method based on self-supervised contrastive learning, including:

[0035] Obtain the time series data to be tested;

[0036] The time series data to be detected is input into the time series anomaly detection model for feature extraction and anomaly judgment, and the anomaly probability or anomaly label of each time window is output. The input time series anomaly detection model includes a feature extraction network and classifier based on A-SCINet. A-SCINet is an SCINet model with an embedded attention mechanism. The feature extraction network based on A-SCINet is pre-trained by the contrastive learning self-supervised learning method, and the pre-trained feature extraction network and classifier based on A-SCINet are trained using pseudo labels.

[0037] Furthermore, training the feature extraction network based on A-SCINet by contrastive learning self-supervised learning method includes:

[0038] Get the original time series;

[0039] Preprocess the original time series and divide it into time windows of fixed length to obtain time segments;

[0040] Select time segments and inject different types of abnormal disturbances to construct positive and negative samples;

[0041] A-SCINet is used to extract features from positive and negative samples, and the extracted features are input into MLP to output feature vectors.

[0042] The feature vector is optimized by contrastive loss function.

[0043] Specifically, the original time series data is preprocessed, including missing value filling, denoising, and normalization, to generate standardized time series data; the standardized time series is then segmented according to a fixed time window to form multiple time segments with overlapping areas as the training sample set;

[0044] Each time slice is input to a feature extraction module, which is based on an A-SCINet architecture formed by a plurality of SCI-Block units recursively stacked to form a full binary tree structure. Each SCI-Block performs odd-even decomposition on the time series and extracts subsequence features using one-dimensional convolution, respectively, and realizes feature interaction through affine transformation. The plurality of SCI-Block layers are connected hierarchically, and local features are fused from bottom to top, and global information is fed back from top to bottom, thereby enhancing the modeling capability of the time series multi-scale dependency relationship.

[0045] After feature extraction, an embedded adaptive attention mechanism based on A 2 -Nets self-attention network, by calculating the correlation between the features of each time step in the time slice, dynamically weighting the features of each layer, highlighting the key time point features that may contain anomalies. This mechanism not only improves the feature interaction capability of each layer, but also through the nested weighting method, so that the model can more accurately capture local and global anomaly patterns, especially in the detection of complex and weak anomalies.

[0046] After the feature extraction and attention enhancement stage is completed, the high-dimensional multi-scale features obtained still need to be further processed in order to realize more abstract and discriminant feature expression. Therefore, a multi-layer perceptron (MLP) module is introduced to perform nonlinear transformation and high-dimensional mapping on the fused features, further improving the feature discrimination capability and providing more refined feature representation for the classifier.

[0047] Constructing positive and negative sample pairs, using Triplet Loss contrastive loss function to optimize the feature space, ensuring that normal samples are clustered in the feature space and abnormal samples are far away from normal samples.

[0048] Furthermore, different types of abnormal disturbances include: global anomaly, trend anomaly, periodic anomaly, context anomaly and shape anomaly.

[0049] Specifically, in the training samples, multiple types of synthetic anomalies are actively injected, including but not limited to global amplitude mutation, local trend deviation, periodic disturbance, context mutation and shape distortion, to construct difficult negative samples for contrastive learning and enhance the model's anomaly recognition ability.

[0050] Furthermore, feature extraction of positive and negative samples by A-SCINet includes:

[0051] The normalized time series segments in the positive and negative samples are input to the sequence decomposition unit, and the normalized time series segments are smoothed to obtain trend components and residual components;

[0052] The trend components are processed through a linear layer to obtain trend features;

[0053] The residual component is input into the SCI-Block unit of the SCI-Net model for recursive feature extraction, and the recursive features and trend features are combined to obtain the final time series feature representation, wherein each SCI-Block unit is embedded with a dual self-attention network based on A 2 -Nets for dynamic weighting in feature extraction.

[0054] Further, training the pre-trained A-SCINet-based feature extraction network and the classifier includes:

[0055] Feature extraction is performed on the positive and negative samples by the pre-trained A-SCINet-based feature extraction network to obtain feature vectors;

[0056] According to the feature vectors, a nearest neighbor set is obtained from the training set;

[0057] According to the nearest neighbor set, pseudo labels are dynamically generated;

[0058] The pre-trained A-SCINet-based feature extraction network and the classifier are trained by a joint loss function, wherein the joint loss function includes a consistency loss calculated according to the nearest neighbor set, an inconsistency loss calculated according to the farthest neighbor set, and a cross-entropy loss calculated according to the pseudo labels.

[0059] Further, according to the feature vectors, the nearest neighbor set is obtained from the training set, which includes: according to the feature vectors, selecting the nearest neighbors in the feature space from the positive and negative samples according to the K-nearest neighbor principle to obtain the nearest neighbor set.

[0060] Further, according to the nearest neighbor set, the pseudo labels are dynamically generated, which includes: according to the known labels or historical pseudo labels in the nearest neighbor set, the pseudo labels are dynamically generated by majority voting.

[0061] Specifically, based on the trained feature representation, the similarity between each time slice and other samples is calculated, the neighborhood structure of the samples is established, and K nearest neighbors and K farthest neighbors are selected for subsequent pseudo label generation and classifier training.

[0062] In the self-supervised classification stage, the pseudo labels are determined according to the neighborhood relationship, and the classifier is trained using the cross-entropy loss, so that the classifier can judge whether a new time series slice is abnormal or not.

[0063] Further, outputting the anomaly probability or anomaly label of each time window includes:

[0064] The A-SCINet structure output result is subjected to Softmax normalization processing to generate a probability distribution of each time window belonging to a normal or abnormal category, and an abnormal score is calculated according to the probability distribution to determine the abnormality degree of each time window.

[0065] Specifically, the time series feature output of the A-SCINet structure is subjected to Softmax function normalization processing to generate an abnormal probability distribution corresponding to each time window, and a threshold or an abnormal score function is combined to realize accurate anomaly detection of the time series.

[0066] The abnormal score is not only based on the Softmax probability, but also can be combined with the relative distance of samples in the feature space, neighborhood sparsity and other indicators for comprehensive judgment to realize a multi-dimensional anomaly detection strategy and adapt to the differentiated requirements of false positive rate and false negative rate in different scenarios.

[0067] In the model training process, a dynamic early stopping mechanism is introduced to monitor the change of F1 score in real time, and when the F1 score does not significantly improve for a continuous preset number of rounds, the training is stopped to prevent model overfitting.

[0068] After training, the trained model is used to extract features, detect anomalies and output the abnormal probability or abnormal label of each time window for the actual application of anomaly detection tasks.

[0069] This method supports batch offline analysis and real-time online detection, has good module expandability, can adjust the feature extraction layer, attention strategy and classifier structure according to different business requirements, and is suitable for diversified industrial and financial environments.

[0070] The present embodiment will be further described below in conjunction with the accompanying drawings:

[0071] The present embodiment provides a time series anomaly detection method based on self-supervised contrastive learning, as shown in Figure 1 which can specifically include two stages, namely a self-supervised contrastive learning stage and a self-supervised classification stage.

[0072] The self-supervised contrastive learning stage includes data preprocessing, anomaly injection mechanism, feature extraction, embedded attention mechanism, MLP feature fusion, contrastive learning, domain modeling and pseudo-label output.

[0073] Data preprocessing:

[0074] In the process of implementing the present application, first, the original time series data is subjected to systematic preprocessing to improve the adaptability of the model to different data sources and eliminate the interference of noise. The preprocessing includes three parts: missing value filling, denoising and normalization standardization. The normalization formula is as follows: μ and σ are the sample mean and standard deviation, respectively.

[0075]

[0076] The missing value filling can be selected according to the business requirements, for example, for the missing point x t , the average value of the adjacent points on both sides can be used for filling, that is, x t = (x t-1 +x t+1 ) / 2. For noise removal, sliding average or wavelet transform is used to effectively smooth short-time sharp fluctuations. In the aspect of normalization processing, Z-Score standardization is adopted to convert each time series data into a standard distribution with a mean of 0 and a variance of 1. The standardized data facilitates the model to quickly converge during training and reduces the influence of different feature scales on the training results.

[0077] After standardization, the time series is divided into fixed-length time windows, usually 50 to 200 time steps, and is overlapped and divided to form local continuous time segments. This overlapping window design can ensure that the model captures local features while maintaining time-dependent continuity, providing high-quality input for the subsequent feature extraction module.

[0078] Abnormal injection mechanism:

[0079] In the implementation process, in order to overcome the problem of scarcity of abnormal samples in actual time series data, a systematic abnormal injection mechanism is designed. This mechanism simulates a variety of common abnormal patterns in the real world, artificially implants synthetic abnormalities in normal time series samples, and then constructs training samples for comparative learning and self-supervised classification. The entire abnormal injection process is based on sliding time windows, and each time segment is locally modified to generate diversified abnormal features.

[0080] First, the system randomly selects a certain proportion of time windows from the standardized time series samples as injection targets. These windows will be injected with different types of abnormal disturbances in various forms after being selected. The abnormal types include but are not limited to: seasonal abnormalities (Seasonal), trend abnormalities (Trend), global amplitude abnormalities (Global), context-related abnormalities (Contextual), and shape distortion abnormalities (Shapelet), etc. Each type of abnormality is implemented with specific parameter combinations to ensure the diversity and complexity of injected abnormalities.

[0081] For periodic anomalies, the system simulates the scenario where the sensor is subjected to periodic disturbances in real-world conditions by adjusting the cycle frequency of selected subsequences. This process involves the expansion and compression of subsequences and the application of amplitude adjustments to form periodic disturbance patterns. Trend anomalies are injected by gradually adding a linearly increasing or decreasing trend component at the end of the window, simulating the effects of slow drift in system parameters or hardware aging. Global amplitude anomalies are represented by a significant overall mutation in the values within the entire window, usually achieved by multiplying the subsequence by an amplification factor greater than 1. Context-dependent anomalies occur locally in a specific time period or specific dimension, disrupting the continuity of the segment and context through local scaling and disturbance. Shape distortion anomalies simulate local waveform mutations by introducing random disturbances while maintaining the starting point, breaking the natural change pattern of the time series and generating abnormal shapes.

[0082] In the anomaly injection mechanism, the selection of subsequences is random. The system randomly determines the starting position and length of the injected subsequence based on the window length, usually 20%-90% of the total length of the window. In the processing of multi-dimensional time series, the dimension of the injected anomaly is also randomly selected, with the number of dimensions varying between 20%-100% of the total number of features. Each selected dimension is independently injected with one type of anomaly, ensuring that the injected time window exhibits highly diversified abnormal features.

[0083] In addition, to ensure the flexibility and unpredictability of anomaly injection, parameters such as compression factor, scaling factor, trend slope, and noise level are randomly selected within a reasonable range. For example, the scaling factor is randomly selected between 0.1 and 2.0, and the trend factor follows a normal distribution N(1, 0.5) with positive and negative changes. This mechanism dynamically generates abnormal samples in each training period, allowing the model to be exposed to new abnormal patterns during training, enhancing its ability to adapt to complex anomalies and improve its discrimination ability.

[0084] Feature extraction:

[0085] In the implementation of this method, the feature extraction module is based on the A-SCINet architecture, combining sequence decomposition technology and hierarchical interaction learning mechanism, aiming to comprehensively capture local dynamic features and global trend features in time series. First, the standardized time series segment is sent to the sequence decomposition unit. This unit smooths the time series by the MovingAverage method, decomposing the original sequence into trend components and residual components. The operation of moving average calculates the mean value of each time step in the input sequence within the local neighborhood, thus extracting the smooth long-term trend part, while the short-term fluctuations are retained as residual parts to capture the local change characteristics of the sequence.

[0086] The trend component is processed by two linear layers to fit and project it in the time dimension, respectively. The first linear transformation keeps the time dimension consistent and enhances the expression ability of the trend feature, and the second layer maps the trend information to the output length to ensure that the trend feature can match the final features of the residual part.

[0087] The residual part then enters the A-SCINet network in a multi-layer recursive stack. As shown in (a)-(b), the core of the A-SCINet is the SCI-Block unit, and each SCI-Block first performs odd-even decomposition on the input sequence to divide the sequence into two sub-sequences with odd and even indexes. One-dimensional convolution is performed on the two sub-sequences to extract local features. After the convolution process, affine transformation is applied to realize the interactive fusion of the features of the two sub-sequences to capture the nonlinear dependence between them. Multiple SCI-Block units are recursively combined in the form of a full binary tree to hierarchically deepen the model's perception of the multi-scale features of the time series. This hierarchical interaction mechanism integrates local features from the bottom up and introduces global context information from the top down, improving the modeling effect of long-term dependencies. Figure 2

[0088] After completing the recursive feature extraction of the SCI-Block, the model adjusts the output dimension of the residual features through a one-dimensional convolution projection layer and merges them with the trend features to generate the final time series feature representation. In this process, if the RIN mechanism is enabled, the model will also standardize the residual and trend features separately and de-standardize them after feature extraction to further improve the robustness and stability of the model.

[0089] The A-SCINet feature extraction module not only accurately perceives short-term local abnormal fluctuations in time series, but also maintains high sensitivity to long-term trend changes, and the extracted features have good discriminative ability and generalization performance.

[0090] Embedded attention mechanism:

[0091] In the A-SCINet architecture proposed in the embodiments of the present application, the adaptive dual attention mechanism is not independently stacked at the end of the network, but is embedded in each SCI-Block hierarchical interaction module and tightly coupled with the local convolution features after odd-even decomposition. This mechanism draws on the attention structure of A 2 -Nets, and constructs attention weighting channels in the time and feature dimensions for each sub-sequence block.

[0092] In the time dimension, the model constructs a self-attention correlation matrix of the feature sequence to measure the dependence between time steps and generates a dynamic attention weight matrix accordingly:

[0093] ​

[0094] where F i denotes the feature representation at the i-th time step.

[0095] In the channel dimension, the model globally averages the pooling of each dimension of the feature channel, and generates a channel weight ω d to reflect the importance of the channel in the current sub-sequence. The double attention mechanism is embedded in each SCI-Block, enabling the model to respond to abnormal related time points and channel dimensions at each stage of feature extraction.

[0096] Through this "layer-embedded" attention enhancement method, the model can perceive potential abnormal signals layer by layer, improving the robustness and discriminative ability of complex abnormal patterns (such as trend anomalies, periodic variations, and weak mutations). It is significantly better than traditional post-attention mechanisms.

[0097] The layer-embedded approach used in this embodiment has obvious advantages: on the one hand, this mechanism can perceive and strengthen potential abnormal signals in real time during feature extraction, achieving more fine-grained abnormal modeling; on the other hand, the double attention mechanism can adaptively focus on key time steps and key channels at each layer, effectively improving the ability to identify multiple types of anomalies. Experiments show that the layer-attention structure has higher F1-score and robustness performance in identifying weak anomalies, boundary anomalies and other complex scenarios, verifying its superior abnormal perception ability.

[0098] Multi-layer perceptron (MLP) feature integration:

[0099] In the specific implementation process, the features are first flattened to convert the three-dimensional tensor into a two-dimensional input matrix to adapt to the input format of the MLP. The matrix is then sent to the MLP module containing multiple fully connected layers. Each fully connected layer consists of linear transformation, nonlinear activation function, and optional regularization operation.

[0100] The first layer of the MLP performs linear transformation to map the input feature dimension from T·D to the hidden layer dimension H, which is achieved by the following formula:

[0101]

[0102] where W (1) ∈R H×(T·D) is the weight matrix, and b (1)is the bias term, and φ(·) is a nonlinear activation function, usually ReLU or LeakyReLU. The introduction of this activation function helps the model capture complex nonlinear relationships between features and improve the model's ability to express high-dimensional features.

[0103] After the first layer of processing, the intermediate feature h (1) Further passed into the second layer and even more layers of fully connected networks. Each layer still follows the same linear transformation and nonlinear activation structure, extracting higher-order feature representations layer by layer. Usually, the number of hidden layers of MLP is 2 to 3, and the number of hidden units is between 128 to 512, which can be flexibly set according to the complexity of the task.

[0104] In the last layer of MLP, a fixed-dimensional feature vector h (L) ∈R C where C represents the number of categories. In this embodiment, the feature is used for feature distance measurement and optimization in contrast learning.

[0105] Contrast learning:

[0106] The core purpose of contrast learning is to construct feature comparisons between positive and negative samples, and use the Triplet Loss loss function to optimize the feature space structure, so that the model can automatically learn high-discriminative feature representations to distinguish normal and abnormal samples.

[0107] To ensure the effective distribution of samples in the feature space, this embodiment uses the Triplet Loss contrast loss function, whose optimization goal is to minimize the feature distance between the anchor and the positive sample, while maximizing the feature distance between the anchor and the negative sample, and keeping the distance difference between the two at least greater than a preset boundary threshold α. The loss function is defined as follows:

[0108]

[0109] where is the anchor sample, is the positive sample, is the negative sample, and α is the interval threshold. By minimizing this loss function, the model gradually learns how to gather normal samples together in the feature space while moving away from abnormal samples, thereby forming a clear classification boundary.

[0110] Positive samples are selected based on similarity measures in the training set, such as Euclidean distance or cosine similarity, to ensure that the positive samples are as close as possible to the anchor samples in feature space. Negative samples are prioritized for samples with a large distance from the anchor features, especially those injected with complex anomalies, to maximize the distinguishing power of the feature space. Furthermore, to enhance the model's adaptability to various anomaly patterns, triple combinations are continuously refreshed during training, ensuring that the model is constantly exposed to new sample comparison tasks, thereby improving the diversity and generalization of contrastive learning.

[0111] Neighborhood modeling and pseudo-label output:

[0112] After feature space optimization is complete, a neighborhood modeling strategy is used to further enhance the model's classification capabilities. This method calculates the feature distances between each sample and other samples to determine its nearest and furthest neighbor sets. Based on the K-nearest neighbor principle, several nearest neighbor samples are selected and their label distributions are statistically analyzed to generate pseudo-labels for the target sample. These pseudo-labels can effectively guide classifier training in the absence of manual annotation, giving the model greater adaptability.

[0113] The following are the self-supervised classification stages, including data preprocessing and model initialization, dynamic pseudo-label generation, performance evaluation and optimal model update, dynamic early stopping mechanism, anomaly identification and result output.

[0114] Data preprocessing and model initialization:

[0115] During the self-supervised classification phase, the system first loads the data required for training and validation based on the configuration parameters. Through sliding window segmentation and data augmentation, a sample set with rich temporal features is formed. Each sample is composed of normalized time series segments, which are injected using anomaly injection techniques to form a mixed sample containing normal and synthetic anomalies. The loading of training data not only ensures data diversity but also provides the necessary input foundation for the model's subsequent self-supervised training. At the same time, the model is initialized based on the parameters obtained in the previous pre-training phase to fully inherit the multi-scale feature expression capabilities learned earlier. The model is deployed in a multi-GPU environment using the DataParallel mechanism to improve the efficiency of large-scale data processing.

[0116] After model initialization is complete, the optimizer is initialized based on the current model parameters and the learning rate policy specified in the configuration file. If an existing training checkpoint file is detected, the system restores the model's historical state from that file, including training progress, optimal performance metrics, and the classifier's current pseudo-label mapping. This mechanism ensures training continuity and avoids retraining completed sections.

[0117] Dynamic pseudo-label generation:

[0118] In the self-supervised classification stage, the system does not rely on human annotation, but provides adaptive supervision signals for training samples through a dynamic pseudo-label generation strategy. This process is based on the modeling results of the model on the sample distribution in the feature space, dynamically determines the class label of each sample for classifier training.

[0119] First, the model uses the pre-trained SCI-Net and A 2 -Nets combined structure to extract features for all training samples, and maps them to classification feature representations through the connected MLP module to form the final high-dimensional feature vector for subsequent pseudo-label generation. Each time window segment x i After the hierarchical interaction of SCI-Net and the attention enhancement of A 2 -Nets, it is mapped to a feature vector h i ∈R D .

[0120] After the generation of the feature representation h i , the system calculates the feature similarity between each sample and other samples in the positive and negative sample set constructed in the self-supervised contrast learning stage based on the Euclidean distance or cosine similarity, and constructs the neighborhood structure of the sample. For each sample h i , the system selects K nearest neighbors and K farthest neighbors from all training samples in the feature space.

[0121] Subsequently, the following two loss functions are introduced as auxiliary supervision signals:

[0122] Consistency Loss:

[0123]

[0124] Where: φ s (ω) is the predicted probability distribution after softmax, φ s (ω) T ·φ s (ω n ) represents the inner product of the two distributions, and the higher the similarity, the closer to 1.

[0125] is used to enhance the similarity between the current sample and its "nearest neighbor" in the predicted class distribution, so that similar samples obtain the same class prediction result. This loss calculates the inner product similarity of the softmax output between the current sample and its nearest neighbor, and maximizes this similarity in the form of cross-entropy.

[0126] Inconsistency Loss:

[0127]

[0128] to reduce the similarity between the current sample and its "most distant neighbor" in the class prediction distribution, thus promoting the model to form clear distinctions at the decision boundary. This loss enforces the separation between classes by minimizing the similarity between the probability distributions of the current sample and its most distant neighbor. Finally, the training objective of the time series anomaly detection model is jointly composed of three loss functions, the joint loss function L total :

[0129] L total = L consistency - L inconsistency - β · L entropy

[0130] where L entropy is the cross-entropy loss, used to avoid the class prediction from being overly concentrated on a single class and to enhance the diversity of the model's output. The pseudo-label is compared with the model's current output prediction p i via the cross-entropy loss function to guide the update of the model parameters; the parameter β is a hyperparameter used to adjust the weight of this loss in the overall optimization objective.

[0131] For each sample, its pseudo-label is dynamically determined by the known labels or historical pseudo-labels of its nearest neighbor set through majority voting. Specifically, if the majority of the K nearest neighbors are determined to be normal, the pseudo-label of the current sample is normal; otherwise, it is determined to be abnormal. This strategy is based on the feature clustering assumption that similar features should have the same class attribute.

[0132] The pseudo-label generation process is updated once per round, and as the model training is gradually optimized, the discriminability of the feature space is enhanced, making the quality of the pseudo-labels continuously improve.

[0133] Performance evaluation and optimal model update:

[0134] During the training process, the system performs prediction on the training set and the validation set every round of iteration and evaluates the current performance of the model. After the SCI-Net and A 2 -Net joint feature extraction, the model output is input into the MLP layer to generate classification scores (logits), and then normalized by the Softmax layer to obtain the class probability distribution of each time window. During the training process, the system calculates the classification error of the model on the training data based on the cross-entropy loss function, and combines the F1 score of the validation set to monitor the performance of the model in real time.

[0135] After each training round, the system first performs a complete forward propagation on the validation set to obtain the prediction results of the model for all validation samples. Through functions such as get_predictions and classification_evaluate, the system respectively calculates the minimum loss value (lowest_loss), the classification head loss (head_loss), and the F1 score (rep_f1) of the current round. The F1 score, as an important indicator of model classification performance, is the core standard for judging the quality of the model.

[0136] If the F1 score of the current round model on the validation set is greater than the historical best F1 record, the system considers the current model as the optimal state. At this time, the system automatically saves the complete parameters of the current model, including the weights of SCI-Net and A 2 -Nets, the optimizer state, the current round index, the loss information of the current classification head, and the normal label mapping relationship (normal_label).

[0137] The save action is implemented by torch.save, which is divided into temporary checkpoints and long-term preserved best models (classification_model) for easy recovery during training interruption or subsequent inference deployment.

[0138] Dynamic early stopping mechanism:

[0139] To prevent the model from overfitting during the pseudo-label driven training process, especially the excessive fitting of unstable pseudo labels, the system designs and implements a dynamic early stopping mechanism based on performance indicators. This mechanism monitors the classification performance of the model on the validation set in real time, especially the trend of the F1 score, dynamically determines whether the model has reached the optimal convergence state, and thus controls the training rounds to improve the generalization ability and stability of the model.

[0140] Specifically, after each training round, the system calculates the F1 score F1 current of the current validation set, and compares it with the historical best F1 score F1 best . If the current F1 score compared to the last round has an improvement amplitude ΔF1 = |F1 current -F1 last | lower than the preset threshold ∈ (such as 0.001), the system records a round of "no significant performance improvement". This record is achieved through a counter, which continuously tracks the number of consecutive rounds without improvement.

[0141] When the counter accumulates a value exceeding a set "patience" threshold (e.g., 50 rounds), the system considers that the model has converged, triggering an early stopping mechanism. At this time, the training process is suspended, the system saves the current recorded best model parameters, and subsequent training iterations are terminated. This mechanism effectively avoids the model continuing to optimize in the presence of uncertainty in pseudo-labels, leading to prolonged training time and performance degradation.

[0142] Abnormality discrimination and result output:

[0143] After training, the A-SCINet-based feature extraction network outputs the original feature distribution for each time window, which is normalized by the Softmax layer and converted into a standardized abnormal probability value. Specifically, the Softmax function maps the original logits z i output by the model to a probability p i : where p i represents the probability that the current window belongs to the abnormal class.

[0144] The system performs binary judgment on the abnormal probability of each time window according to a pre-set abnormality discrimination threshold θ (e.g., 0.5). If p i > θ, it is determined to be abnormal; otherwise, it is considered normal. This threshold can be flexibly adjusted according to the specific application scenario to balance the false positive rate and the false negative rate.

[0145] Finally, the system outputs the discrimination result and the corresponding abnormal score for each time window. The results can be exported as CSV, Excel files, or directly called by downstream application systems for real-time monitoring, early warning, or further analysis. The entire abnormality discrimination process not only guarantees efficient adaptive learning of unannotated time series, but also improves the practicality and stability of the model in complex environments through refined probability scoring.

[0146] The above-described embodiments are only descriptions of the preferred modes of the present application and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those of ordinary skill in the art shall fall within the scope of protection determined by the claims of the present application.

Claims

1. A time series anomaly detection method based on self-supervised contrastive learning, characterized in that: include: Obtain the time series data to be tested; The time series data to be detected is input into the time series anomaly detection model for feature extraction and anomaly judgment, and the anomaly probability or anomaly label of each time window is output, wherein the input time series anomaly detection model includes a feature extraction network and a classifier based on A-SCINet, and the A-SCINet is an SCINet model with an embedded attention mechanism. The feature extraction network based on A-SCINet is pre-trained by a comparative learning self-supervised learning method, and the pre-trained feature extraction network based on A-SCINet and the classifier are trained by pseudo labels.

2. The time series anomaly detection method based on self-supervised contrastive learning according to claim 1, characterized in that Training the feature extraction network based on A-SCINet by contrastive learning self-supervised learning method includes: Get the original time series; Preprocessing the original time series and dividing it into time windows of fixed length to obtain time segments; Select the time segment and inject different types of abnormal disturbances to construct positive and negative samples; Perform feature extraction on the positive and negative samples through the A-SCINet, input the extracted features into the MLP, and output feature vectors; The feature vector is optimized by a contrastive loss function.

3. The time series anomaly detection method based on self-supervised contrastive learning according to claim 2, characterized in that: Different types of abnormal disturbances include: global anomalies, trend anomalies, periodic anomalies, contextual anomalies, and shape anomalies.

4. The time series anomaly detection method based on self-supervised contrastive learning according to claim 2, characterized in that: Extracting features from the positive and negative samples using the A-SCINet includes: Inputting the standardized time series segments in the positive and negative samples into a sequence decomposition unit, performing smoothing processing on the standardized time series segments to obtain a trend component and a residual component; Processing the trend component through a linear layer to obtain trend features; The residual component is input into the SCI-Block unit of the SCINet model for recursive feature extraction, and the recursive feature and the trend feature are merged to obtain the final time series feature representation, wherein each SCI-Block unit is embedded with an A-based 2 -Nets' dual self-attention network for dynamic weighting in feature extraction.

5. The time series anomaly detection method based on self-supervised contrastive learning according to claim 2, characterized in that: Training the pre-trained A-SCINet-based feature extraction network and the classifier using pseudo labels includes: Extract features from the positive and negative samples using a pre-trained A-SCINet-based feature extraction network to obtain feature vectors; Obtaining a nearest neighbor set and a farthest neighbor set from the positive and negative samples according to the feature vector; Dynamically generate pseudo labels based on the nearest neighbor set; The pre-trained A-SCINet-based feature extraction network and the classifier are trained through a joint loss function, wherein the joint loss function includes a consistency loss calculated according to the nearest neighbor set, an inconsistency loss calculated according to the farthest neighbor set, and a cross entropy loss calculated according to the pseudo label.

6. The time series anomaly detection method based on self-supervised contrastive learning according to claim 5, characterized in that: Obtaining a nearest neighbor set and a farthest neighbor set according to the feature vector includes: selecting, according to the feature vector and the K-nearest neighbor principle, several closest neighbor and farthest neighbor samples in the feature space from the positive and negative samples to obtain the nearest neighbor set and the farthest neighbor set.

7. The time series anomaly detection method based on self-supervised contrastive learning according to claim 5, characterized in that: Dynamically generating pseudo labels according to the nearest neighbor set includes: dynamically generating the pseudo labels by majority voting according to known labels or historical pseudo labels in the nearest neighbor set.

8. The time series anomaly detection method based on self-supervised contrastive learning according to claim 1, characterized in that: Outputting the anomaly probability or anomaly label for each time window includes: The output results of the feature extraction network based on A-SCINet are subjected to Softmax normalization processing to generate a probability distribution of whether each time window belongs to the normal or abnormal category, and the anomaly score is calculated based on the probability distribution to determine the degree of abnormality of each time window.

Images