CLIP-based zero sample time sequence anomaly detection method and system

Abstract

The invention discloses a CLIP-based zero sample time sequence anomaly detection method and system, and the method combines time sequence data and corresponding labels or descriptive information, and carries out the time sequence anomaly detection through a zero sample learning mode. The method comprises the following steps: firstly, preprocessing time series data, extracting key features, and then, carrying out embedded mapping on the features and text description by utilizing a CLIP model, so as to realize effective characterization of the time series data. And then, comparing and analyzing with historical data, and recognizing potential abnormal points or abnormal behaviors by combining the similarity between text semantics and a time sequence mode. The method does not need to depend on a large amount of labeled data, can efficiently process the anomaly detection problem of unseen samples, and has relatively strong generalization ability and robustness. The method has high theoretical property and application value, and is suitable for time sequence anomaly detection tasks in multiple fields of industry, finance, medical treatment and the like.

Description

Technical Field

[0001] The present invention relates to a time series data analysis method, and in particular to a CLIP-based zero-sample time series anomaly detection method and system, which is suitable for time series data anomaly detection and prediction in the fields of industry, finance, and medicine. Background Art

[0002] With the continuous development of information technology, time series data has been widely used in various industries, especially in industrial monitoring, financial analysis, healthcare, and other fields. Anomaly detection in time series data is an important means to ensure system stability and security. However, traditional time series anomaly detection methods often rely on manually designed features or rules or require large amounts of labeled data to train anomaly detection models. This is not feasible in many practical scenarios, especially when there is a lack of sufficient labeled data.

[0003] In recent years, breakthroughs in deep learning technology, particularly the successful application of pre-trained models, have provided new insights for anomaly detection in time series data. CLIP (Contrastive Language-Image Pretraining), a cross-modal pre-training model, embeds images and text into a unified feature space, enabling zero-shot learning between images and text. This feature enables CLIP to excel in a variety of application scenarios, particularly with its high robustness when processing cross-modal data (such as images and text).

[0004] In time series data anomaly detection, time series data can be processed by rendering it into images and inputting it into CLIP. This not only breaks the traditional method's reliance on manual feature extraction, but also effectively utilizes the advantages of zero-shot learning to solve the problem of lack of labeled data.

[0005] While several deep learning-based time series anomaly detection methods exist, most still rely on large amounts of labeled data and typically require high computational resources during model training. In contrast, the CLIP-based zero-shot learning method leverages comparative learning between images and text, achieving more efficient anomaly detection that is independent of large amounts of labeled data. This method not only improves anomaly detection accuracy but also reduces data requirements during training, demonstrating its significant practical value. Summary of the Invention

[0006] The purpose of this paper is to provide a zero-sample time series anomaly detection method based on CLIP, which aims to accurately detect anomalies in time series without the need for labeled data by combining time series data and image data through comparative learning.

[0007] To achieve the above-mentioned object of the invention, the present invention adopts the following technical solution: a zero-sample time series anomaly detection method based on CLIP, comprising the following steps:

[0008] (1) Pre-training the CLIP model and obtaining a training set containing image data and text data, where the text data contains text information describing these images;

[0009] (2) Using an image encoder to convert image data into image features, and using a text encoder to convert text data into text features;

[0010] (3) Pairing image features with text features, with positive samples being images and text describing their content, and negative samples being images and irrelevant text. The overall CLIP model is trained through contrastive learning. This model can embed images and text into the same feature space, thereby achieving zero-shot learning between images and text through contrastive learning.

[0011] (4) performing periodic segmentation on the time series data for which anomaly detection is required, wherein the period is obtained by analyzing the frequency domain characteristics of the time series data or determined by setting a predefined fixed value;

[0012] (5) Convert each segment of time series data into a two-dimensional matrix structure and render it into a two-dimensional time series image, where each dimension of the image represents a specific dimension of data in the time series;

[0013] (6) The two-dimensional time series image is input into the trained CLIP model. The CLIP model outputs whether there is an anomaly in the time series data based on the comparative learning of image features and text features, and further outputs the type of anomaly. The anomaly type includes mutation anomaly of pixel mutation, periodic anomaly of image stripe spacing, or trend deviation of image brightness gradient trend.

[0014] Furthermore, the training part in step (3) also includes:

[0015] During training, the image encoder and text encoder minimize the contrast loss between image features and their corresponding text features, making the image and text as close as possible in the embedding space, thereby achieving cross-modal representation learning;

[0016] Through contrastive learning, the CLIP model can not only understand the image content, but also understand the meaning of the time series data represented by the image through the corresponding text, thereby supporting zero-sample time series anomaly detection.

[0017] Furthermore, the processing of time series data includes:

[0018] Use Fast Fourier Transform (FFT) to obtain the frequency domain information of time series data, determine the periodicity of time series data, and thus perform appropriate time series data segmentation;

[0019] Each dimension in the generated two-dimensional time series image represents a variable or feature of the time series data, and the other dimension represents the time step, ensuring that the data conversion process can effectively reflect the dynamic changes of the time series data.

[0020] Furthermore, the processing of the two-dimensional time series image data includes:

[0021] Standardize and normalize the generated two-dimensional time series images to meet the input requirements of the CLIP model;

[0022] Before inputting data, the time series images are preprocessed by image enhancement to increase the robustness of the model to different data transformations;

[0023] The standardized two-dimensional time series images are input into the CLIP model. The CLIP model uses a comparative learning mechanism to determine whether there are abnormalities in the input time series images and can identify the specific type of abnormalities.

[0024] Furthermore, the data input also includes:

[0025] Each time series data segment is segmented using a sliding window to generate multiple time series segments. Each time series segment is then periodically segmented and arranged to obtain a corresponding two-dimensional time series image, thereby improving the model's ability to learn the diversity of time series data and time series features.

[0026] The two-dimensional time series image corresponding to each time slice is input into the CLIP model. The model will perform anomaly detection and analysis on the data of each time period and finally give anomaly prediction results.

[0027] On the other hand, the present invention also provides a zero-sample time series anomaly detection system based on CLIP, which includes:

[0028] The data acquisition module is used to obtain continuous time series data from the time series data source;

[0029] The image rendering module is used to periodically segment the time series data and render each segment of time series data into a two-dimensional time series image to ensure that the image can reflect the dynamic changes in the data;

[0030] The pre-training module is used to perform comparative learning training through image encoders and text encoders to obtain a pre-trained CLIP model that can process image and text data;

[0031] The anomaly detection module is used to input the generated two-dimensional time series image into the CLIP model. The CLIP model outputs whether there is an anomaly, determines the type of anomaly and classifies it.

[0032] Furthermore, the data acquisition module includes:

[0033] It is used to obtain continuous time series data from sensors, real-time data acquisition systems or historical data sets, and provide it to subsequent image rendering modules for further processing.

[0034] Furthermore, the image rendering module includes:

[0035] It is used to slice time series data through a sliding window to generate multiple time slices, and render each time slice into a two-dimensional time series image according to the period of the time series data, further enhancing data processing capabilities;

[0036] The rendered two-dimensional time series images are input into the CLIP model for processing.

[0037] Furthermore, the pre-training module includes:

[0038] A CLIP model based on contrastive learning is trained. Images and texts are trained by minimizing the contrastive loss, so that the similarity between images and texts is calculated in the same embedding space, thereby supporting zero-shot learning.

[0039] Furthermore, the anomaly detection module includes:

[0040] The two-dimensional time series image is input into the CLIP model. The model learns by comparing the image and text to determine whether the time series data contains anomalies, and outputs the anomaly type based on the comparison between the image features and the text description.

[0041] Compared with the existing technology, the CLIP-based zero-sample time anomaly detection method provided by the present invention has the following advantages:

[0042] (1) Unsupervised learning. The method of the present invention does not rely on manually labeled data. Through zero-sample learning, it can be widely applied to various industrial time series data, greatly reducing the cost of manual labeling.

[0043] (2) High precision and generalization. By using large-scale image datasets (such as ImageNet) for pre-training, the CLIP model can learn a strong correspondence between images and text, so that when time series images are input into the model, it can obtain efficient anomaly detection capabilities and is applicable to different types of time series data.

[0044] (3) Efficiency and real-time performance. Compared with traditional time series data processing methods, the method of the present invention can effectively reduce computational complexity and perform anomaly detection on time series data in real time, thus meeting the real-time requirements in industrial scenarios.

[0045] (4) Strong adaptability. By combining images and text for comparative learning, the present invention can automatically adapt to different time series data types and abnormal patterns, improving the generalization ability and adaptability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall module framework diagram of the present invention;

[0047] Figure 2 It is a CLIP model pre-training flow chart of the present invention;

[0048] Figure 3 is a one-dimensional time series data processing flow chart of the present invention;

[0049] Figure 4 Schematic diagram of the two-dimensional temporal image input CLIP model of the present invention. DETAILED DESCRIPTION

[0050] The specific implementation method and working principle of the present invention are described in detail below with reference to the accompanying drawings:

[0051] The present invention provides a zero-shot time series anomaly detection method based on CLIP. By combining the feature visualization processing of time series data with a zero-shot learning strategy of image-text comparative learning, it can effectively detect anomalies in time series without manually annotated data.

[0052] The overall module framework diagram of the method of the present invention is as follows Figure 1 As shown, the description is as follows:

[0053] The framework consists of four main modules: data acquisition module, image rendering module, pre-training module, and anomaly detection module. Each module is connected through the data flow sequence to jointly complete the anomaly detection task of time series data.

[0054] The CLIP model pre-training process of the present invention is as follows Figure 2 As shown, the description is as follows:

[0055] Step (1): Preparation of image and text data

[0056] During the pre-training phase, we first need to prepare image and text data. In order for the CLIP model to learn the relationship between images and text through contrastive learning, we need to select appropriate images from public datasets (such as ImageNet) and provide corresponding text descriptions for these images. The specific operation should ensure that:

[0057] a. Select a large amount of image data from datasets such as ImageNet. These images should cover a variety of objects, scenes, and dynamic features to ensure diversity.

[0058] b. Provide a corresponding text description for each image. These text descriptions can be short labels such as "cat" or "dog", or longer natural language descriptions that include a detailed explanation of the image content.

[0059] The correspondence between images and text provides a basis for contrastive learning, enabling the CLIP model to learn the similarities and differences between images and text.

[0060] Step (2): Image encoder and text encoder extract features

[0061] Next, the image encoder and text encoder learn the multimodal features between the image and the text. The image encoder is usually based on the architecture of a convolutional neural network (CNN) or a visual transformer (ViT), which processes the input image and extracts its high-dimensional features. The goal of the encoder is to convert the image into a feature vector representation of a fixed dimension that can capture the visual information in the image. The text encoder is usually based on the Transformer architecture, with the input being a text description (such as a natural language description of the image) and the output being a feature vector representation of the text. The goal of the text encoder is to map the text description to the same feature space as the image, so that similar images and texts are closer in the space.

[0062] Step (3): Contrastive learning optimization

[0063] Contrastive learning is the core of CLIP pre-training, which is optimized through the following steps:

[0064] a. Encode each pair of image and text to obtain the feature vector of the image and the feature vector of the text.

[0065] b. Calculate the similarity between the image feature vector and the text feature vector (usually using cosine similarity).

[0066] c. By maximizing the similarity of identical image and text pairs and minimizing the similarity of different image and text pairs, the parameters of the image encoder and text encoder are optimized so that similar images and texts are closer in the feature space, while dissimilar images and texts are separated.

[0067] d. Through multiple rounds of training, the CLIP model can effectively learn the joint representation of images and text, enabling the model to process various types of image and text data.

[0068] e. Construct a multimodal comparative learning feature library. Taking the text modality as an example, in the field of anomaly detection, the constructed library contains word combinations that describe different abnormal and normal states. These can be mapped into text embeddings of different categories. This can be used to calculate the similarity between a new input image and existing text, thereby performing zero-shot classification or retrieval.

[0069] Through contrastive learning, the image encoder and text encoder are optimized simultaneously, so that identical or similar image-text pairs have higher similarity in the feature space, while dissimilar image-text pairs have lower similarity.

[0070] Step (4): Validation and tuning of the pre-trained model

[0071] After the pre-training phase is complete, the CLIP model will be validated and tuned to ensure that it can effectively process multimodal data. This process includes:

[0072] a. Validate on multiple benchmark datasets (such as ImageNet, COCO, etc.) to evaluate the performance of the model.

[0073] b. Optimize the model for time series tasks to improve the generalization ability of the model.

[0074] Before inputting the time series data into the pre-trained CLIP model, the one-dimensional time series data needs to be preprocessed to make it a two-dimensional time series image that meets the CLIP input requirements. The preprocessing process of time series data is as follows: Figure 3 The specific description is as follows:

[0075] Step (1): The original one-dimensional time series data obtained from the data source (such as a sensor, database, or real-time acquisition system) is used as input. The one-dimensional time series data usually contains continuous timestamps and corresponding measurement values, which are used to describe the dynamic behavior of the system or device over time.

[0076] Step (2): Split the time series by period P. Split the one-dimensional time series data by the specified period P, and divide the original time series data into multiple time periods. The period P can be determined by spectrum analysis (such as fast Fourier transform FFT) or preset according to business needs. Each segment of the data after segmentation contains P continuous sampling points, which can preserve the periodic characteristics of the time series data.

[0077] Step (3): Splicing of time series segments. Arrange the segmented time series segments in sequence and splice them into a two-dimensional matrix. Specifically, assume that n segments of time series data are obtained after segmentation (respectively, "time series segment 1", "time series segment 2" to "time series segment n"), and use each segment of data as a row (or column) of the two-dimensional matrix, ultimately forming an n×P two-dimensional matrix, where each element in the matrix represents the measurement value at the corresponding moment.

[0078] Step (4): Bilinear interpolation expansion of the two-dimensional matrix. To meet the input size requirements of the pre-trained CLIP model, the two-dimensional matrix generated by the above splicing is expanded to the target image size (e.g., 224×224) through bilinear interpolation. Bilinear interpolation performs weighted calculations on the pixels in the matrix to ensure that the expanded image has continuity and retains the dynamic characteristics of the original data.

[0079] Step (5): Render the interpolated and expanded two-dimensional matrix into a two-dimensional time series image. During the rendering process, the data values ​​in the matrix are represented by grayscale or pseudo-color, so that the image can intuitively show the dynamic change characteristics of the time series data. The generated two-dimensional time series image meets the input requirements of the CLIP model and can be used as input data for the subsequent anomaly detection module.

[0080] like Figure 4 As shown in the figure, the CLIP model is used for time series anomaly detection as follows:

[0081] Step (1): Input 2D time series image

[0082] The pre-processed 2D time series image is fed into the pre-trained CLIP model. This image is generated by periodic segmentation, concatenation, and bilinear interpolation of the time series data, fully reflecting the dynamic characteristics of the time series data.

[0083] Step (2): Extract features using the CLIP model. The CLIP model extracts features from the input 2D time series image and compares and matches them with predefined text features in the model. The CLIP model uses the joint contrastive learning capabilities of the image encoder and text encoder to determine whether the image data conforms to the feature distribution of normal samples.

[0084] Step (3): Determine whether an anomaly exists. The CLIP model determines whether the input two-dimensional time series image contains abnormal features based on the results of feature comparison. If the image features match the feature distribution of normal samples to a high degree, the current input is considered a "normal sample"; if the match is low, an anomaly is considered to exist, and the specific type of anomaly is further analyzed.

[0085] Step (4): Output the detection results.

[0086] a. If there is no abnormality: The CLIP model outputs the detection result as "normal sample", indicating that the current time series data has no abnormal changes within the monitoring range.

[0087] b. If an anomaly exists: The CLIP model outputs the detection result as the specific type of anomaly, which may include but is not limited to "mutation anomaly", "periodic anomaly" or "trend deviation". For example, after being converted into an image, the "mutation anomaly" in the time series may appear as a pixel mutation in the local area that is out of tune with the surrounding image; "periodic anomaly" (such as an increase in period) may appear as an elongation of the spacing between stripes in the image; and "trend deviation" may manifest as an image that originally darkens from left to right, and suddenly begins to brighten after a certain position. The discrimination of anomaly types is based on the multimodal contrastive learning feature library constructed by the CLIP model during the pre-training stage. Based on the comparative learning of images and texts during the training process, the CLIP model can perform zero-sample anomaly detection without manual labeling and can automatically identify different types of anomalies.

[0088] Through the above steps, the present invention implements time series anomaly detection based on the CLIP model. This method can extract features from the original time series and efficiently and accurately represent the state of the original time series data by converting it into two dimensions. Furthermore, this detection method combines the capabilities of image and text comparative learning, significantly improving the model's anomaly detection and adaptability to complex time series data.

[0089] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The method embodiments described above are merely illustrative. For example, the division of steps is merely a logical function division, and other division methods may be used in actual implementation. Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0090] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included in the scope of protection of the present application.

Claims

1. A zero-sample time series anomaly detection method based on CLIP, characterized in that: The following steps are involved: (1) Pre-training the CLIP model and obtaining a training set containing image data and text data, where the text data contains text information describing these images; (2) Using an image encoder to convert image data into image features, and using a text encoder to convert text data into text features; (3) Pairing image features with text features, with positive samples being images and text describing their content, and negative samples being images and irrelevant text. The overall CLIP model is trained through contrastive learning. This model can embed images and text into the same feature space, thereby achieving zero-shot learning between images and text through contrastive learning. (4) performing periodic segmentation on the time series data for which anomaly detection is required, wherein the period is obtained by analyzing the frequency domain characteristics of the time series data or determined by setting a predefined fixed value; (5) Convert each segment of time series data into a two-dimensional matrix structure and render it into a two-dimensional time series image, where each dimension of the image represents a specific dimension of data in the time series; (6) The two-dimensional time series image is input into the trained CLIP model. The CLIP model outputs whether there is an anomaly in the time series data based on the comparative learning of image features and text features, and further outputs the type of anomaly. The anomaly type includes mutation anomaly of pixel mutation, periodic anomaly of image stripe spacing, or trend deviation of image brightness gradient trend.

2. The method according to claim 1, characterized in that The training part in step (3) also includes: During training, the image encoder and text encoder minimize the contrast loss between image features and their corresponding text features, making the image and text as close as possible in the embedding space, thereby achieving cross-modal representation learning; Through contrastive learning, the CLIP model can not only understand the image content, but also understand the meaning of the time series data represented by the image through the corresponding text, thereby supporting zero-sample time series anomaly detection.

3. The method according to claim 1, characterized in that Time series data processing includes: Use Fast Fourier Transform (FFT) to obtain the frequency domain information of time series data, determine the periodicity of time series data, and thus perform appropriate time series data segmentation; Each dimension in the generated two-dimensional time series image represents a variable or feature of the time series data, and the other dimension represents the time step, ensuring that the data conversion process can effectively reflect the dynamic changes of the time series data.

4. The method according to claim 1, wherein The processing of two-dimensional time series image data includes: Standardize and normalize the generated two-dimensional time series images to meet the input requirements of the CLIP model; Before inputting data, the time series images are preprocessed by image enhancement to increase the robustness of the model to different data transformations; The standardized two-dimensional time series images are input into the CLIP model. The CLIP model uses a comparative learning mechanism to determine whether there are abnormalities in the input time series images and can identify the specific type of abnormalities.

5. The method according to claim 1, wherein The data input also includes: Each time series data segment is segmented using a sliding window to generate multiple time series segments. Each time series segment is then periodically segmented and arranged to obtain a corresponding two-dimensional time series image, thereby improving the model's ability to learn the diversity of time series data and time series features. The two-dimensional time series image corresponding to each time slice is input into the CLIP model. The model will perform anomaly detection and analysis on the data of each time period and finally give anomaly prediction results.

6. A CLIP-based zero-sample time series anomaly detection system, characterized by: The system includes: The data acquisition module is used to obtain continuous time series data from the time series data source; The image rendering module is used to periodically segment the time series data and render each segment of time series data into a two-dimensional time series image to ensure that the image can reflect the dynamic changes in the data; The pre-training module is used to perform comparative learning training through image encoders and text encoders to obtain a pre-trained CLIP model that can process image and text data; The anomaly detection module is used to input the generated two-dimensional time series image into the CLIP model. The CLIP model outputs whether there is an anomaly, determines the type of anomaly and classifies it.

7. The system according to claim 6, characterized in that The data acquisition module includes: It is used to obtain continuous time series data from sensors, real-time data acquisition systems or historical data sets, and provide it to subsequent image rendering modules for further processing.

8. The system according to claim 6, wherein: The image rendering module includes: It is used to slice time series data through a sliding window to generate multiple time slices, and render each time slice into a two-dimensional time series image according to the period of the time series data, further enhancing data processing capabilities; The rendered two-dimensional time series images are input into the CLIP model for processing.

9. The system according to claim 6, wherein: The pre-training module includes: A CLIP model based on contrastive learning is trained. Images and texts are trained by minimizing the contrastive loss, so that the similarity between images and texts is calculated in the same embedding space, thereby supporting zero-shot learning.

10. The system according to claim 6, wherein: The anomaly detection module includes: The two-dimensional time series image is input into the CLIP model. The model learns by comparing the image and text to determine whether the time series data contains anomalies, and outputs the anomaly type based on the comparison between the image features and the text description.

Images