A method for detecting timing anomalies based on differential components
By combining and expanding time-series data and extracting features based on differential components, and combining them with deep learning networks, the problem of classification accuracy and efficiency with few samples in time-series anomaly detection is solved, and efficient classification detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANXI UNIV
- Filing Date
- 2023-04-28
- Publication Date
- 2026-07-10
Smart Images
Figure CN116561685B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of deep learning technology, specifically relating to a temporal anomaly detection method based on differential components, which can be used in temporal anomaly detection. Background Technology
[0002] Early deep learning technologies achieved remarkable results in image classification, pattern recognition, and object tracking. However, these advancements heavily relied on manually labeled data. In specific applications, such as time series anomaly detection, art classification, and the medical field, the difficulty in obtaining samples, the limited sample size, and the challenges of labeling significantly increased the time and manpower required, severely hindering the development of deep learning classification capabilities. Specifically, due to the limited sample size and overly complex networks in some tasks, directly applying deep learning techniques to these tasks can easily lead to overfitting, i.e., high training accuracy but low test accuracy. Choosing appropriate deep learning models for small sample sizes can not only reduce training costs and shorten training time but also allow complex models to be applied to new categories, thereby expanding the application scope of existing models. For example, in art classification tasks, some styles of paintings are rare due to historical reasons, resulting in a limited number of samples available for classification training. This makes it difficult for existing deep learning techniques to obtain an effective classification model to identify unknown samples. This necessitates the development of data processing methods and classification learning models specifically for small sample sizes.
[0003] There are already many studies on few-shot classification, mainly including classification methods based on transfer learning, data augmentation, and metric-based classification methods.
[0004] Classification methods based on transfer learning primarily involve first training a basic network on a large, labeled dataset, and then fine-tuning the model parameters on a smaller dataset. The advantage of transfer learning-based classification methods is that pre-training on a large dataset followed by fine-tuning on a smaller dataset can yield good classification results. The disadvantage is that when there are significant class differences between images in the dataset, the model's classification accuracy decreases.
[0005] Data augmentation-based classification methods typically expand the sample size by preprocessing the data, such as rotating or transforming it. The advantages of data augmentation-based classification methods are that they can increase the amount of data in the dataset to some extent, alleviating overfitting problems that occur in few-shot learning. The disadvantages are that the relatively small overall amount of labeled data limits the methods of data augmentation; while they can improve training performance to some extent, they cannot completely solve the overfitting problem.
[0006] Metric-based methods primarily rely on convolutional neural networks (CNNs). Image features are extracted using CNNs, and sample categories are predicted based on the model's metric rules and the distance or similarity between sample classes. Compared to the previous two methods, metric-based classification methods can learn quickly and effectively. However, metric-based classification methods do not perform well when dealing with complex samples containing a large amount of information.
[0007] In time series data, the distribution of anomalous data and normal data is often highly imbalanced. Therefore, time series anomaly detection can be regarded as a classification problem in the case of imbalanced datasets. When using deep learning algorithms to learn and train on it, due to the huge difference in the number of anomalous data and normal data, the classifier does not pay enough attention to the samples and cannot learn effective features. The classification results tend to be majority-oriented.
[0008] Furthermore, time series anomaly detection requires handling datasets with limited labeled data, necessitating data augmentation. However, due to the dependencies and non-stationarity of time series data, current data augmentation methods do not fully utilize these inherent characteristics. Therefore, how to perform anomaly detection on small, labeled time series datasets remains a challenge.
[0009] In summary, one pressing issue that researchers in this field need to address is how to ensure both accuracy and efficiency in temporal anomaly detection when only a small amount of sample data is available. Summary of the Invention
[0010] This invention overcomes the shortcomings of existing technologies and aims to solve the following technical problem: providing a small sample classification method based on differential components, which can be applied to image classification and time series anomaly detection, and improve the accuracy and efficiency of classification.
[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a time-series anomaly detection method based on differential components, comprising the following steps:
[0012] S1. Obtain labeled time series data samples, select some samples as fixed type samples, and obtain samples of the same type and different types to form training set and test set; the samples in the training set and test set are not repeated, and the number of samples of the same type and different types in each set is the same;
[0013] S2. Combine and expand the data samples in the training set and the test set using fixed-type samples to obtain combined sequence samples;
[0014] S3. Input the combined sequence data samples of the expanded training set and test set into the differential component to train and test the differential component; the differential component includes:
[0015] Feature extraction unit: used to extract features from each sample in the combined sequence samples in the dataset to obtain feature maps;
[0016] Similarity calculation unit: It calculates the distance similarity of each sample in the learnable kernel combination sequence sample by channel to obtain the distance features of each channel;
[0017] Differential feature calculation unit: calculates the differential features of the combined sequence data samples by using the distance features of each channel;
[0018] S4. Input the differential feature map output by the differential component into the feedforward network for training to obtain the temporal anomaly classifier;
[0019] S5. Combine the time series data to be tested with fixed type samples to obtain the type samples to be tested. Use the trained differential component to obtain the differential feature map of the type samples to be tested. Input it into the time series anomaly classifier to obtain the type of the time series data to be tested.
[0020] Preferably, in step S2, the method for expanding the data sample is as follows:
[0021] Each time series sample in each set is combined with each sample in the fixed type to obtain a combined sequence sample containing two samples.
[0022] The label of combined sequence samples of the same class is set to 1, and the label of combined sequence samples of different classes is set to 0.
[0023] Preferably, in step S3, the similarity calculation unit performs distance similarity calculation to obtain distance feature Z. i The calculation formula is:
[0024]
[0025] Among them, z i (u,v) represents the distance feature Z. i The feature value at position (u, v), where (x, y) represents the pixel position coordinates. ω represents the pixel value at position (x, y) in channel i of a randomly selected item r. (u-x,v-y) Let represent the parameter value at position (ux, vy) in the learnable kernel W corresponding to the randomly selected term r, and t represent the term other than r. ω represents the pixel value at position (x,y) in channel i of term t. (u-x',v-y)Let t represent the parameter value at position (ux,vy) in the learnable kernel W' corresponding to the term t, where b is the bias. Here, m represents the number of channels, and n represents the number of items in the combined sample.
[0026] Preferably, in step S3, the difference component updates the parameters of the learnable kernel using the cross-entropy loss function L, the expression of which is:
[0027]
[0028] Where y represents the actual label. This indicates the predicted label.
[0029] Preferably, in step S3, the difference feature calculation unit obtains the difference feature calculation formula through distance feature calculation as follows:
[0030]
[0031] Where U represents the difference feature, q represents the distance coefficient, and Z... i This represents the distance feature map of channel i.
[0032] Preferably, in step S2, the method for expanding the data samples is as follows: each time series sample in each set is combined with a samples from a fixed type of sample to obtain a combined sequence sample including a+1 samples; where a is greater than or equal to 2.
[0033] In step S3, the method by which the similarity calculation unit calculates the distance features of each channel is as follows:
[0034] Randomly select a sample, calculate its distance to the remaining samples using the distance formula, and take the mean of the results as the distance feature of that channel.
[0035] Preferably, in step S4, the classifier used by the feedforward network is a ResNet classifier or an MLP classifier.
[0036] Preferably, in step S1, a portion of the normal time-series data samples is selected as fixed-type samples, another portion of the samples is selected as samples of the same type, and then a portion of the abnormal time-series data samples is selected as out-of-type samples, with the number of samples of the same type and out-of-type samples being the same.
[0037] This invention proposes a temporal anomaly detection method based on discriminative components. First, the sample data is preprocessed, and new learning objects are constructed from multiple samples through permutation and combination. Second, discriminative components are established, and weight learning is used to reduce the differences between similar samples and increase the differences between dissimilar samples. The component results are then passed to a deep learning network, enabling significant differentiation of input features in subsequent networks. Finally, the deep learning network is used to complete the classification task. This invention has the following advantages compared to existing technologies:
[0038] (1) New data augmentation method: By using permutation and combination, the original independent samples are transformed into related samples, which expands the number of samples and enables the model to learn the relationship between multiple samples.
[0039] (2) The present invention uses differential components to extract differential features of combined samples. The differential components can be combined with various deep learning techniques to improve the accuracy of classification and enable its application in few-sample learning.
[0040] (3) This invention effectively solves the problem of unsatisfactory classification results among complex data in current few-shot learning;
[0041] (4) This invention is simple, practical and has high classification accuracy. It does not require long-term training and can efficiently achieve classification and detection tasks. It can also be applied to other few-sample classification problems such as painting classification. Attached Figure Description
[0042] Figure 1 A simplified structural diagram of a timing anomaly detection method based on differential components provided in an embodiment of the present invention;
[0043] Figure 2 This is a schematic diagram of the data augmentation and preprocessing structure in an embodiment of the present invention;
[0044] Figure 3 This is a simplified structural diagram of the differential components in an embodiment of the present invention;
[0045] Figure 4 This diagram illustrates the accuracy of a time-series anomaly detection method based on differential components in a single training iteration using two different classifiers, as provided in an embodiment of the present invention. The horizontal axis represents the number of iterations, and the vertical axis represents the accuracy. Detailed Implementation
[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0047] Example 1
[0048] like Figure 1 As shown, Embodiment 1 of the present invention provides a method for detecting temporal anomalies based on differential components, comprising the following steps:
[0049] Step 1: Obtain time series data samples;
[0050] Step 2: Expand the sample. Given a time series dataset, which includes 300 normal samples and 300 abnormal samples, expand the data in this dataset.
[0051] In this embodiment, 30 sequences are randomly selected from the entire normal time-series data sample as known samples, referred to as known fixed-type samples. Simultaneously, 270 sequences are selected from this sample as samples of the same type. Then, 270 sequence samples are randomly selected from the entire abnormal data sample as samples of the opposite type. The samples of the same type and the samples of the opposite type together constitute the sample set, serving as the test type sample. To ensure the reliability of training, samples of the same type and the opposite type are randomly selected without repetition from the 270 test type samples in a 5:1:4 ratio, and these are set as training set samples, validation set samples, and test set samples. Finally, the test samples in each set are combined with the fixed-type samples. Two input samples of the same type are labeled 1, and conversely, two input samples of different types are labeled 0, such as... Figure 2 As shown in the figure, the number of sample combinations with label 0 is equal to the number of sample combinations with label 1.
[0052] Step 3: Input the combined sequence data samples of the expanded training set and test set into the differential component to train and test the differential component.
[0053] In this embodiment, the structure of the differential component is as follows: Figure 3As shown, it borrows the principle of weight sharing in convolutional neural networks, using the nonlinear distance between corresponding receptive fields of different samples as the final feature obtained by the model. Here, Ix and I_y represent the two samples included in the input combination sample; r_x, g_x, and b_x represent the RGB channels of the I_x sample; r_y, g_y, and b_y represent the RGB channels of the I_y sample; W_x and W_y represent the learnable kernels corresponding to the I_x and I_y samples; r1_x, g1_x, and b1_x represent the feature values of each channel of the Ix sample; r1_y, g1_y, and b1_y represent the feature values of each channel of the I_y sample; Z_1, Z_2, and Z_3 represent the distance features of the corresponding channels of the combination sample obtained by calculating the distance; and Dis represents the output of the difference component: difference features.
[0054] Therefore, in this embodiment, the differentiating component includes:
[0055] Feature extraction unit: used to extract features from each sample in the combined sequence samples in the dataset to obtain feature maps;
[0056] Similarity calculation unit: used to calculate the distance similarity of each sample in the combined sequence sample through a learnable kernel, and obtain the distance features of each channel;
[0057] Differential feature calculation unit: The differential features of the combined sequence data samples are calculated by the distance features of each channel.
[0058] Specifically, in this embodiment, a learnable kernel is used to extract the feature values of each sample. The learnable kernel is slid across all channels of the corresponding time series data I to obtain the feature map C of the m channels of the corresponding time series data. i W is used to calculate the distance feature, where i ranges from (1, m), and m is the number of channels in the time series data. Specifically, the similarity calculation unit performs distance similarity calculation on the feature maps of each i-channel to obtain the distance feature Z. i The formula is:
[0059]
[0060] Among them, z i (u,v) represents the distance feature Z. i The feature value at position (u, v), where i represents the channel, r represents a randomly selected item, and (x, y) represents the pixel coordinates. ω represents the pixel value at position (x, y) in channel i of a randomly selected item r. (u-x,v-y) Let represent the parameter value at position (ux, vy) in the learnable kernel W corresponding to the randomly selected term r, and t represent the term other than r. ω represents the pixel value at position (x,y) in channel i of item t. (u-x',v-y) Let represent the parameter value at position (ux,vy) in the learnable kernel W' corresponding to term t, where b is the bias. is the activation function. n represents the number of items in a combined sample (e.g., n is 2 for pairwise combinations), and an item is a time series data point in the combined sample.
[0061] Specifically, in this embodiment, the parameters of the learnable kernel W are updated using the cross-entropy loss function, where y represents the true label. This indicates the predicted label.
[0062]
[0063] Specifically, in this embodiment, the similarity metric is calculated using Euclidean distance to obtain the output of the difference component: difference features:
[0064]
[0065] Where U represents the difference feature, q represents the distance coefficient, and Z... i This represents the distance feature map of channel i.
[0066] Table 1. Accuracy of three distance measurement methods
[0067]
[0068] Table 1 shows the accuracy under various measurement methods. As can be seen from Table 1, the Euclidean distance used in this embodiment to calculate the difference feature has the highest accuracy.
[0069] In this embodiment, the difference feature map reflects the distance between the corresponding receptive fields of different samples. The difference feature obtained from combinations of samples of the same type is labeled 1, while the difference feature obtained from combinations of dissimilar samples is labeled 0. The difference component uses sample similarity metrics to determine the magnitude of the differences between individual samples and learns the nonlinear relationships between samples. Therefore, the difference component proposed in this invention can effectively identify whether input samples belong to the same type, thereby completing the classification task. When faced with data samples of unknown types, combining them with time-series data of known types can also determine whether they belong to an existing type.
[0070] In addition, as another implementation method, in this embodiment, the method for expanding the data samples is as follows: each time series sample in each set is combined with a samples in a fixed type sample to obtain a combined sequence sample including a+1 samples; where a is greater than or equal to 2.
[0071] The method by which the similarity calculation unit calculates the distance features of each channel is as follows:
[0072] Randomly select a sample, calculate its distance to the remaining samples using the distance formula, and take the mean of the results as the distance feature of that channel.
[0073] Step 4: Input the differential features obtained from the differential components into a deep learning-based feedforward network for training to complete the anomaly detection task and obtain a temporal anomaly classifier.
[0074] Because of the discriminant differences between the two samples obtained from the discriminant component, fine-tuning of the deep learning network is necessary. The classifiers selected in this invention are mainly MLP and ResNet, and the experimental results in one training iteration are as follows: Figure 4 As shown in Table 2, the results of these two classifiers sufficiently demonstrate the advantage of the differential features output by the differential component of this invention in most classification algorithms.
[0075] Table 2 Accuracy of Differential Components and Indifferent Components
[0076]
[0077] Step 5: Combine the time series data to be tested with fixed type samples to obtain the type samples to be tested. Use the trained differential component to obtain the differential feature map of the type samples to be tested. Input it into the time series anomaly classifier to obtain the label of the type samples to be tested, and further obtain the type of the time series data to be tested.
[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting temporal anomalies based on differential components, characterized in that, Includes the following steps: S1. Obtain labeled time series data samples, select some samples as fixed type samples, and obtain samples of the same type and different types to form training set and test set; the samples in the training set and test set are not repeated, and the number of samples of the same type and different types in each set is the same; the time series data is image data. S2. Combine and expand the data samples in the training set and the test set using fixed-type samples to obtain combined sequence samples; S3. Input the combined sequence data samples of the expanded training set and test set into the differential component to train and test the differential component; the differential component includes: Feature extraction unit: used to extract features from each sample in the combined sequence samples in the dataset to obtain feature maps; Similarity calculation unit: It performs distance similarity calculation on each sample in the learnable kernel-based combined sequence sample by channel to obtain the distance features of each channel; the channels include RGB channels; Differential feature calculation unit: calculates the differential features of the combined sequence data samples by using the distance features of each channel; S4. Input the differential feature map output by the differential component into the feedforward network for training to obtain the temporal anomaly classifier; S5. Combine the time series data to be tested with fixed type samples to obtain the type samples to be tested. Use the trained differential component to obtain the differential feature map of the type samples to be tested. Input it into the time series anomaly classifier to obtain the type of the time series data to be tested. In step S3, the similarity calculation unit performs distance similarity calculation to obtain the distance feature Z. i The calculation formula is: in, Representing distance features The feature value at position (u, v), where (x, y) represents the pixel position coordinates. This represents the pixel value at position (x, y) in channel i of a randomly selected item r. This indicates that the learnable kernel W corresponding to the randomly selected item r ( The parameter value at position t represents a term other than r. This represents the pixel value at position (x, y) in channel i of term t. In the learnable kernel W' corresponding to term t ( The position parameter value, where b is the offset. Here, m represents the number of channels, and n represents the number of items in the combined sample.
2. The method for detecting temporal anomalies based on differential components according to claim 1, characterized in that, In step S2, the method for expanding the data sample is as follows: Each time series sample in each set is combined with each sample in the fixed type to obtain a combined sequence sample containing two samples. The label of combined sequence samples of the same class is set to 1, and the label of combined sequence samples of different classes is set to 0.
3. The method for detecting temporal anomalies based on differential components according to claim 1, characterized in that, In step S3, the differential component updates the parameters of the learnable kernel using the cross-entropy loss function L, the expression of which is: ； Where y represents the actual label. This indicates the predicted label.
4. The method for detecting temporal anomalies based on differential components according to claim 1, characterized in that, In step S3, the difference feature calculation unit obtains the calculation formula for the difference feature through distance feature calculation, which is: ； in, Representing the difference feature, q represents the distance coefficient, and Z represents the difference feature. i This represents the distance feature map of channel i.
5. The method for detecting temporal anomalies based on differential components according to claim 1, characterized in that, In step S2, the method for expanding the data samples is as follows: each time series sample in each set is combined with a samples from a fixed type sample to obtain a combined sequence sample including a+1 samples; where a is greater than or equal to 2. In step S3, the method by which the similarity calculation unit calculates the distance features of each channel is as follows: Randomly select a sample, calculate its distance to the remaining samples using the distance formula, and take the mean of the results as the distance feature of that channel.
6. The method for detecting temporal anomalies based on differential components according to claim 1, characterized in that, In step S4, the classifier used by the feedforward network is either a ResNet classifier or an MLP classifier.
7. The method for detecting temporal anomalies based on differential components according to claim 1, characterized in that, In step S1, a portion of the normal time-series data samples is selected as fixed-type samples, another portion of the samples is selected as samples of the same type, and then a portion of the abnormal time-series data samples is selected as out-of-type samples. The number of samples of the same type and out-of-type samples is the same.