A time series anomaly detection method based on a visual language model
By using a visual language model to model the multi-scale features of time series, the problem of insufficient generalization ability and low computational efficiency in existing methods is solved, and efficient and accurate multi-scale anomaly detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2026-05-09
- Publication Date
- 2026-06-05
AI Technical Summary
Existing time series anomaly detection methods rely on a large amount of labeled data, have insufficient generalization ability, are difficult to effectively model multi-scale anomaly features, and have large computational costs, making it difficult to balance detection accuracy and efficiency.
A visual language model-based approach is adopted, which transforms time series into multi-scale visual feature maps through temporal decomposition and multi-channel image processing. Feature extraction and verification are then performed by combining a visual encoder and a visual language model to achieve multi-scale anomaly detection.
It improves the accuracy and efficiency of anomaly detection, enhances the model's generalization ability and the interpretability of results, and reduces computational overhead.
Smart Images
Figure CN122156834A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent operation and maintenance and data analysis technology, and in particular relates to a time series anomaly detection method based on a visual language model. Background Technology
[0002] With the rapid development of large-scale information systems, the Industrial Internet, and intelligent manufacturing, the scale of data generated during system operation is exploding. This data typically records the system's operational status at different time dimensions in time series format. By analyzing time series data, abnormal behaviors during system operation can be detected in a timely manner, which is of great significance for fault diagnosis, system maintenance, and risk warning. Therefore, time series anomaly detection has become an important research direction in the fields of intelligent operation and maintenance and data analysis.
[0003] Existing time series anomaly detection methods mainly include those based on statistical models and those based on machine learning or deep learning. Traditional statistical methods typically rely on prior assumptions about the data distribution, resulting in insufficient adaptability when dealing with complex, non-stationary, and variable time series data. With the development of deep learning, researchers have proposed anomaly detection methods based on models such as recurrent neural networks, autoencoders, and Transformers. These methods can characterize the nonlinear features and temporal dependencies in time series to a certain extent. However, these methods usually rely on large amounts of labeled data or require training on specific datasets. When the data distribution changes or the application scenario shifts, the model often needs to be retrained, leading to limited generalization ability. Furthermore, anomaly patterns in time series often have multi-scale characteristics, including local abrupt changes and long-term trend shifts. Existing methods are mostly based on fixed time windows or single scales for modeling, making it difficult to effectively capture anomaly features at different scales simultaneously, thus affecting detection accuracy. Using a single-scale feature extraction approach has certain limitations: while short time windows are beneficial for identifying local anomalies such as spikes and sudden drops, they easily overlook anomalies spanning long time periods, such as slow drifts and trend shifts; while longer time windows can retain more global contextual information, they may weaken the salience of local anomalies in the overall sequence, leading to inaccurate anomaly boundary localization or even missed detections. Some existing methods also suffer from high computational overhead and low inference efficiency during detection, especially when dealing with long sequences, large-scale monitoring points, or real-time detection tasks, often making it difficult to balance detection accuracy with practical deployment efficiency.
[0004] In recent years, with the development of visual language models, they have demonstrated excellent performance in cross-modal feature understanding and reasoning. Some studies have attempted to convert time-series data into image form and use visual models for feature extraction to achieve anomaly detection. However, existing methods typically only provide simple visualization of the original time series, failing to fully exploit the structured information such as trends, periods, and residuals within the time series, resulting in limited feature representation capabilities. Furthermore, these methods often rely on fixed-scale image input during anomaly detection, making it difficult to simultaneously address the detection needs of both local fine-grained anomalies and global trend anomalies. Visual models also often face a trade-off between image resolution and temporal context when processing time-series images: when the input window is small, although image details are clearer, sufficient global context is lacking; when the input window is large, image compression can easily reduce the discernibility of local anomalies. Therefore, how to effectively model multi-scale features of time series without requiring a large amount of labeled data, while balancing detection accuracy and computational efficiency, has become a pressing technical problem in the field of time-series anomaly detection. Summary of the Invention
[0005] To overcome the problems of existing technologies in time series anomaly detection, such as reliance on large amounts of labeled data, insufficient generalization ability, and difficulty in effectively modeling multi-scale anomaly features, this invention proposes a time series anomaly detection method based on a visual language model, which improves the accuracy and efficiency of anomaly detection.
[0006] The anomaly detection task of this invention is mainly aimed at univariate time series data in industrial interconnection scenarios. In specific applications, the univariate time series can be understood as continuous monitoring data from an actual system, such as the sampling sequence of a sensor during the operation of industrial equipment. Each time point corresponds to a sampling moment, and the values in the sequence represent the actual physical quantity at that moment, such as equipment vibration acceleration, temperature, or current signal. These values reflect the change process of the equipment's operating state over time. When the equipment malfunctions or operates abnormally, the above physical quantities usually exhibit sudden changes, drifts, or abnormal fluctuations. Therefore, the identification and location of abnormal states can be achieved through the analysis of this time series.
[0007] To achieve the above objectives, the technical solution of this application includes the following steps:
[0008] In a first aspect, this invention proposes a time series anomaly detection method based on a visual language model, comprising the following steps:
[0009] S1. Obtain the time series of the target system's operating status, perform standardization processing to obtain standardized time series data; divide the processed sequence into multiple subsequences using a sliding window, and perform time series decomposition on each subsequence to obtain multiple component sequences; convert each subsequence and its corresponding component sequences into two-dimensional line graphs, and stack them in the channel dimension to generate a multi-channel time series image;
[0010] S2. Input the multi-channel temporal image into the pre-trained visual encoder to extract the basic visual feature map composed of features from multiple image patches; perform pooling processing on the basic visual feature map at different scales to obtain a set of multi-scale visual feature maps.
[0011] S3. Based on a set of multi-scale visual feature maps, multi-scale anomaly score maps are generated through cross-window patch feature matching and similarity calculation; the multi-scale anomaly score maps are fused and mapped to the original time series to generate a one-dimensional anomaly score sequence, thereby locating the preliminary anomaly candidate intervals.
[0012] S4. Convert the complete standardized time series data into a two-dimensional line graph and label the preliminary anomaly candidate intervals. Combined with the preset prompt words, input the labeled line graph and anomaly candidate interval information into the pre-trained visual language model. The visual language model will verify and correct the anomaly candidate intervals and output a formatted result containing the final anomaly interval, confidence level and anomaly description text.
[0013] Furthermore, the operating status time series is a monitoring data sequence of industrial equipment sensors.
[0014] Further, in S1, each subsequence after division is decomposed into time series components, and trend components, seasonal components, and residual components are extracted. The original data of each subsequence, as well as the trend components and seasonal components obtained from the decomposition, are mapped into two-dimensional line charts of uniform size. The two-dimensional line charts do not contain coordinate axes or legend information. The three line charts are stacked in the channel dimension to generate a three-channel time series image.
[0015] Furthermore, the pre-trained visual encoder is a model based on the Vision Transformer architecture, and the patch-level features of the penultimate layer output of the visual encoder are extracted as the basic visual feature map.
[0016] Furthermore, the process of performing pooling at different scales on the basic visual feature map specifically involves:
[0017] The basic visual feature map is subjected to average pooling operation using pooling kernels of different sizes with a pooling step size of 1. Contextual information from different spatial neighborhoods is aggregated to obtain the corresponding multi-scale visual feature map set.
[0018] Furthermore, in S3, the step of generating a multi-scale anomaly score map through cross-window patch feature matching and similarity calculation includes:
[0019] For each patch feature in the visual feature map of the current sliding window at the current scale, calculate its cosine similarity with all patch features in the visual feature maps of all other sliding windows at the same scale.
[0020] Take the maximum cosine similarity value among all the calculation results, subtract the maximum value from 1, and use the result as the anomaly score of the patch feature to obtain the anomaly score map at this scale.
[0021] Further, in S3, the fusion of multi-scale anomaly score maps and mapping to the original time series generates a one-dimensional anomaly score sequence, thereby locating preliminary anomaly candidate intervals, including:
[0022] Upsample the anomaly score maps at each scale to the same spatial resolution as the base visual feature maps;
[0023] Harmonic averaging is performed on the upsampled multi-scale anomaly score maps to obtain a unified two-dimensional anomaly score map;
[0024] Based on the correspondence between image columns in multi-channel time series images and time steps in the original time series, the anomaly scores of all spatial locations corresponding to each time point in the two-dimensional anomaly score image are aggregated, and a one-dimensional anomaly score sequence is generated by calculating a specified quantile.
[0025] Based on a preset discrimination threshold, all consecutive time periods in which the abnormal scores exceed the threshold are extracted from the one-dimensional abnormal score sequence as preliminary abnormal candidate intervals.
[0026] Furthermore, in S4, the preset prompt word contains instructions for guiding the visual language model to perform the following operations:
[0027] Based on the global time series trend and continuity in the input labeled two-dimensional line chart, the preliminary abnormal candidate intervals are verified and intervals that conform to the overall trend, seasonality, or normal fluctuations are eliminated.
[0028] Supplement the identification of intervals that were missed due to weak visual features, but which disrupt the temporal continuity, deviate from the global trend, or have obvious statistical anomalies;
[0029] Output the final anomaly range, corresponding confidence level, and anomaly description text in JSON format.
[0030] Secondly, the present invention proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned time series anomaly detection method based on a visual language model.
[0031] Thirdly, the present invention provides a computer electronic device, including a memory and a processor;
[0032] The memory is used to store computer programs;
[0033] The processor is configured to implement the above-described time series anomaly detection method based on a visual language model when executing the computer program.
[0034] The beneficial effects of this invention are:
[0035] This invention transforms one-dimensional time-series data into a visual representation containing multi-dimensional structured information such as trends and seasons by performing temporal decomposition and multi-channel visualization, thereby enhancing the feature extraction capabilities of subsequent visual models. A pre-trained general visual encoder extracts multi-scale visual features, and unsupervised cross-window feature matching is used for initial anomaly screening. This allows for preliminary localization of multi-scale anomalies without training on specific data, significantly improving the model's generalization ability and adaptability. Finally, a visual language model is introduced to perform global contextual verification and correction of the preliminary results. Leveraging its powerful cross-modal understanding capabilities and combined with complete temporal trend information, it effectively eliminates local false positives and supplements visually inconspicuous true anomalies, thus significantly improving the interpretability and reliability of the results while ensuring high detection accuracy. This invention effectively overcomes the limitations of traditional time-series anomaly detection methods in terms of generalization ability, multi-scale feature modeling, result interpretability, and computational efficiency. Attached Figure Description
[0036] Figure 1 This is a schematic diagram of the framework of a time series anomaly detection method based on a visual language model;
[0037] Figure 2 This is a flowchart illustrating a time series anomaly detection method based on a visual language model.
[0038] Figure 3 A schematic diagram of the time series data preprocessing and visualization process;
[0039] Figure 4 This is a schematic diagram illustrating the decomposition of a time series into trend components, seasonal components, and residual components.
[0040] Figure 5 A schematic diagram of a two-dimensional line graph of a time series and the annotation of candidate anomaly intervals;
[0041] Figure 6 This is a schematic diagram of a computer electronic device provided by the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.
[0043] This invention proposes a time-series anomaly detection method based on a visual language model. For example... Figure 1 As shown, this method first preprocesses the input time-series data, segmenting it using a sliding window and employing a time-series decomposition method to extract trend, seasonal, and residual components, enhancing the ability to express the structured information of the time series. Subsequently, the original sequence and its decomposed components are converted into multi-channel time-series images. A visual encoder extracts patch-level features, and multi-scale feature extraction is used to model anomaly patterns at different time scales, thus achieving preliminary localization of anomaly candidate intervals. Based on this, the cross-modal understanding and reasoning capabilities of a visual language model are further combined to verify, supplement, and correct the preliminarily located anomaly candidate intervals, outputting the final anomaly detection result. This invention can effectively model multi-scale features of time series without requiring training on specific data, ensuring both accuracy and computational efficiency in anomaly detection.
[0044] like Figure 2 As shown, a time series anomaly detection method based on a visual language model mainly includes the following steps:
[0045] Step 1: Obtain the time series of the target system's operating status, perform standardization processing to obtain standardized time series data; divide the processed sequence into multiple subsequences using a sliding window, and perform time series decomposition on each subsequence to obtain multiple component sequences; convert each subsequence and its corresponding component sequences into two-dimensional line graphs, and stack them in the channel dimension to generate a multi-channel time series image.
[0046] In this embodiment, as Figure 3 As shown, suppose the univariate time series to be detected is... The univariate time series consists of sampling points arranged in chronological order, derived from sensor monitoring data from industrial equipment. First, the time series data to be detected needs to be standardized using min-max standardization, with the specific formula as follows:
[0047] (1)
[0048] in, , These represent the maximum and minimum values in the input sequence, respectively. After standardization, the sequence values are mapped to a uniform range, thereby reducing the differences in numerical scales between different data points and facilitating subsequent image generation and feature extraction. Then, a sliding window is used to segment the input time-series data, resulting in multiple fixed-length subsequence segments. The sliding window size... Set to 200, with a sliding step of 50. Overlapping areas are preserved between adjacent windows, allowing the same anomaly pattern to be retained across multiple windows. Then, as... Figure 4 As shown, trend, seasonality, and residual component information of subsequence fragments are extracted using time-series decomposition techniques. Let the subsequence fragment data be... The time series decomposition operation can be represented as:
[0049] (2)
[0050] in, The components representing the long-term trend of a time series reflect the overall upward or downward trend. It represents periodic seasonal components and is used to describe recurring, regular changes. This represents the residual after removing the trend and seasonal components. The original data, trend components, and seasonal components are then mapped to two-dimensional line charts, with a uniform image size of 224×224 pixels and a resolution of 100 dpi. Non-core elements such as axes and legends were removed during line chart generation. Finally, the three images are stacked along the channel dimension to obtain a three-channel input image. .
[0051] Step 2: Input the multi-channel temporal image into the pre-trained visual encoder to extract the basic visual feature map composed of features from multiple image patches; perform pooling processing on the basic visual feature map at different scales to obtain a set of multi-scale visual feature maps.
[0052] In this embodiment, the three-channel image obtained in step 1 is input into a CLIP visual encoder employing a ViT-B / 16 structure. The visual encoder performs patch-level feature extraction on the temporal image, dividing the input image into multiple non-overlapping image patches and outputting the corresponding high-dimensional feature representations. To preserve more complete fine-grained visual information, this invention uses the penultimate patch-level output of the visual encoder as the feature map. In this embodiment, the input image size is 224×224, the patch size is 16×16, therefore P is 14, and the feature dimension of a single patch is D=768, i.e. The final output layer focuses more on global semantic alignment, while the penultimate layer retains richer local structural information, making it more suitable for anomaly detection tasks. Based on this, average pooling with different kernel sizes and a fixed stride of 1 is used to generate a multi-scale feature map set. The formula for calculating pooling operations is:
[0053] (3)
[0054] Where k represents the pooling kernel size, which belongs to the preset pooling size set K; This indicates that the top left corner is the current position. Local neighborhood, These are the row and column offsets within the pooling kernel, used to traverse all patch locations covered by the pooling kernel; These represent the row and column spatial positions in the feature map after pooling. In the original feature map N, the coordinates are... The patch corresponding to the first 3D eigenvalues; For scale Below, the feature map after pooling is located at... The first 3D eigenvalues; Indicates the feature embedding dimension. This represents the feature dimension for a single patch. In this embodiment, the preset pooling kernel sizes are 2 and 3, used to construct medium-scale and large-scale features, respectively. Smaller-scale pooling results can retain more local detail information, making them suitable for identifying short-term anomalies such as spikes, sudden drops, and local mutations; larger-scale pooling results can aggregate a wider range of contextual information, making them suitable for identifying cross-time-step anomalies such as continuous oscillations, trend drift, and level shifts.
[0055] Finally, the feature map after pooling is obtained. This feature map aggregates By analyzing the feature information of neighboring patches within the range, we can obtain a wider range of contextual information, thereby effectively identifying cross-patch anomaly patterns.
[0056] Step 3: Based on the multi-scale visual feature map set, generate a multi-scale anomaly score map by cross-window patch feature matching and similarity calculation; fuse the multi-scale anomaly score map and map it to the original time series to generate a one-dimensional anomaly score sequence, thereby locating the preliminary anomaly candidate interval.
[0057] In this embodiment, after multi-scale feature extraction of time-series data, anomaly scores are calculated based on cross-patch similarity. The anomaly detection approach is based on the assumption that anomalous data constitutes a small proportion of the overall data. In time-series data of the same category or batch, normal patterns typically exhibit high repetition, while anomalous patterns occur less frequently and differ significantly from most normal patterns. Based on this characteristic, the location of anomalies can be initially determined by comparing the similarity between the current patch features and other patch features. During initial anomaly localization, the model compares each patch feature of the current time window with the patch features of all other windows within the same category. Since anomalies occur relatively infrequently, patch segments that differ significantly from most patterns can be identified as anomalous.
[0058] For each sliding window and scale ,remember Indicates the current window In scale The multi-scale patch feature map below is obtained by pooling the original feature map N in step 2. This is for the sliding window... The For each patch to be anomaly scored, it is necessary to calculate the patch's performance relative to other windows at the same scale. All patches The cosine dissimilarity is calculated using the following formula:
[0059] (4)
[0060] in, Indicates the index of the current sliding window; Indicates the index of other sliding windows under the same pooling kernel. ; This indicates the pooling kernel size corresponding to multi-scale feature extraction; Indicates the current window In scale The one-dimensional index of the patch in the feature map below; Indicates reference window In scale The one-dimensional index of the patch in the feature map is used to traverse the window. All patch locations; Display window In scale Next The feature vector of each patch; Display window In scale Next The feature vector of each patch; For cosine similarity, The cosine dissimilarity is the ratio of the number of patches to the number of patches. The larger the value, the greater the difference in features between the two patches. Indicates a single window Find the match with the smallest difference from the current patch among all patches in the current patch; For the current patch With window The cosine dissimilarity value of the best-matching patch. If the difference between the current patch and the best-matching patch is still large, it indicates that the patch is dissimilar to all other patterns and is more likely to be an anomaly.
[0061] For feature maps Each patch p in the image undergoes the above calculation, ultimately resulting in a map that matches the feature map. A grid map with the same spatial size, where the value at each position in the grid is the anomaly score of that patch, is the anomaly score map for the current window i at scale k. At different scales, a set of anomaly score maps at each scale can be obtained. Then, the anomaly score maps at each scale are upsampled to the base patch resolution to unify the anomaly score maps at different scales to the same size. Finally, the anomaly scores at multiple scales are fused by harmonic averaging to obtain a single patch-level anomaly score map.
[0062] Since the horizontal column direction of the image corresponds to the time axis of the time series, while the vertical row direction only represents the spatial features of the image and does not correspond to temporal information, the anomaly score corresponding to each column patch is mapped to the corresponding time step of the original time series according to the time interval to which it belongs within the sliding window. Multiple anomaly scores will be obtained at the same time step due to overlapping sliding windows and multiple rows of patches in the same column. After aligning all patch scores by time step, a two-dimensional anomaly score map M covering the complete time series is finally generated.
[0063] To obtain the one-dimensional anomaly score at each time point, it is necessary to take a specified quantile in the spatial dimension of the two-dimensional anomaly score map to obtain the score at each time point. Corresponding abnormal scores The calculation formula is:
[0064] (5)
[0065] in, This represents the time step index in the original time series data; Representing a two-dimensional anomaly graph In the middle, the time dimension is The set of anomaly scores for all spatial locations; This represents the quantile function, used to calculate the quantile of the input data. quantiles; These are preset quantile parameters. A threshold for discrimination is selected based on a one-dimensional anomaly score sequence. and set it as The Gaussian quantiles. This is a hyperparameter set to 0.01, and the threshold is set to the 99th percentile of a Gaussian distribution. The model will extract all outlier scores exceeding the threshold. The continuous time period is used as the time series anomaly interval for the initial judgment of the model.
[0066] Step 4: Convert the complete standardized time series data into a two-dimensional line chart and label the preliminary anomaly candidate intervals; combine the preset prompt words, input the labeled line chart and anomaly candidate interval information into the pre-trained visual language model, and the visual language model will verify and correct the anomaly candidate intervals, and output a formatted result containing the final anomaly interval, confidence level and anomaly description text.
[0067] In this embodiment, the standardized complete time series data is first... Convert to a two-dimensional line chart, where the horizontal axis represents continuous and complete time step indices, and the vertical axis represents the globally standardized signal values. A light blue semi-transparent shading is used in the line chart to precisely mark the anomaly candidate intervals determined in step 3, ensuring that the shaded area is strictly aligned with the start and end time scales of the anomaly intervals. Figure 5 As shown, the pre-set visual language model prompts, annotated 2D line graphs, and anomaly candidate interval indices are input into the GPT-4o visual language model. Guided by the prompts, the model verifies each candidate anomaly interval based on global temporal trends, temporal continuity, and statistical anomaly characteristics. False positive intervals with local visual anomalies but conforming to the overall trend and normal fluctuations are eliminated, while genuine anomaly intervals missed due to weak visual features are supplemented and identified. The model assigns a confidence score of 1 to 3 to each finally determined anomaly interval according to the strength of the anomaly evidence and generates an anomaly description of no more than 100 characters, outputting the set of anomaly intervals, corresponding confidence arrays, and an explanation of the anomaly cause in standardized JSON format. Finally, the model output is post-processed to filter out low-confidence intervals with a confidence score of 1, retaining only the anomaly intervals with confidence scores of 2 and 3 as the final result of this time series anomaly detection.
[0068] Through the above steps, the visual language model uses the global context information provided by the complete temporal image to verify and correct the candidate anomaly intervals obtained in step 3. This can reduce false positives and false negatives caused by calculations based solely on local feature similarity, and improve the accuracy and interpretability of anomaly detection results.
[0069] In one specific embodiment of the present invention, an optional visual language model prompt word is as follows:
[0070] "You are an expert in both time series analysis and multimodal (visual + verbal) reasoning, possessing a deep understanding of the temporal continuity, trend consistency, and statistical anomalies of time series data. You will acquire two types of information:"
[0071] 1. A two-dimensional polyline chart with annotations completed.
[0072] Horizontal axis: Time step index (accurate to a single time point, with no overlapping or missing ticks);
[0073] Vertical axis: Signal value changing over time (reflecting real-time fluctuations and overall trends);
[0074] The light blue semi-transparent shaded areas in the image represent the preliminary "visual inspection" anomaly windows (each shaded area corresponds to an interval in the preliminary list).
[0075] 2. Preliminary "visual inspection" results
[0076] A list of intervals detected by coarse-grained detection that only focuses on local visual patterns and does not consider global temporal context;
[0077] This range may include false positives (local anomalies that conform to the overall trend) and false negatives (statistical / contextual anomalies that are not visually obvious, such as slow drift or subtle abrupt changes).
[0078] Your goal is to fuse two types of information: a complete visual curve (global trend and temporal continuity) and a preliminary window (local visual features), to output a precisely optimized anomaly detection result for the entire sequence. Specific requirements:
[0079] Eliminate all preliminary ranges that appear abnormal only in certain local areas but conform to the overall trend, seasonal pattern, or normal fluctuations;
[0080] Accurately supplement intervals that are missed in the preliminary visual detection results but disrupt the temporal continuity, deviate from the global trend, or have obvious statistical anomalies (such as sharp peaks / drops, abrupt shifts, slow drifts, and outliers that do not conform to historical patterns).
[0081] The response format is to return a JSON object containing only the following fields:
[0082] 1. "interval_index": An array of [start, end] index pairs for each detected exception (including the first and last indices, accurate to the nearest horizontal axis tick), in the format [[start1, end1], [start2, end2], …], returning [] if no exception is found;
[0083] 2. "confidence": An array of 1-3 integers that correspond one-to-one with the intervals, determined based on confidence levels (global + local).
[0084] 1 = Low confidence: vague or very slight bias (approximately 50%-70% confidence; only marked when there is no reasonable explanation);
[0085] 2 = Medium confidence level: Local anomalies are obvious, with slight global uncertainty (approximately 70%-95% confidence; consistent with local anomalies but slightly in line with the global pattern);
[0086] 3 = High confidence: Strong statistical / contextual anomaly evidence exists (>95% confidence; clearly violates global trend / temporal continuity, with significant visual / statistical anomalies); if no anomaly is found, return [].
[0087] 3. "abnormal_description": A text of no more than 100 words summarizing the reasons for the interval anomaly, focusing on the deviation of the overall trend, the interruption of the time series, or the statistical anomaly (avoid vague descriptions).
[0088] Additional notes:
[0089] Accurately estimate interval boundaries based on the horizontal axis scale, with an error ≤ 1 time step;
[0090] Initial segments may appear unusual due to data truncation. Do not mark them unless there is clear and conclusive evidence of anomalies (such as sharp spikes / drops that do not conform to subsequent trends).
[0091] No additional fields, comments, or formatting should be added; only the JSON object described above should be returned.
[0092] Those skilled in the art can adjust the above prompts according to actual needs.
[0093] To verify the effectiveness and practicality of the method proposed in this invention, the following experimental procedure was constructed for verification.
[0094] (1) Experimental data: This invention uses publicly available time series anomaly detection datasets for testing, namely NASA spacecraft telemetry datasets (including MSL and SMAP subsets).
[0095] All data includes timestamps and corresponding observations. Anomaly labels were not used in model training during the experiment and were only used for result evaluation.
[0096] (2) Comparison Methods: This invention selects several representative time series anomaly detection methods for comparison, mainly including:
[0097] 1. Traditional statistical method: ARIMA;
[0098] 2. Deep learning methods: LSTM, LSTM-AE, TadGAN, AER;
[0099] 3. Temporal pre-trained basic models: UniTS, TimesFM, TimesFM2;
[0100] 4. Visual language modeling methods: TAMA, VLM4TS.
[0101] (3) Evaluation index: The present invention uses the maximum F1 score as the evaluation index, which takes into account both precision and recall, and can comprehensively measure the overall detection performance of the model.
[0102] (4) Hyperparameter settings: All experiments were conducted on a server equipped with an NVIDIA 3090-24G GPU and CUDA version 12.1.1, and the PyTorch version used was 2.4.0. The specific hyperparameter settings of the model are shown in Table 1.
[0103] Table 1 Hyperparameter Settings
[0104]
[0105] (5) Experimental procedure:
[0106] 1. Normalize the original time series data and segment it using a sliding window method;
[0107] 2. Perform time-series decomposition on the data for each time window and convert it into a multi-channel image representation;
[0108] 3. Input the image into a visual encoder to extract features and construct multi-scale features;
[0109] 4. Calculate anomaly scores based on feature similarity to obtain preliminary anomaly candidate intervals;
[0110] 5. The candidate regions are further discriminated and filtered using a visual language model to obtain the final anomaly detection results;
[0111] 6. Compare all test results with the actual abnormal intervals and calculate the evaluation index.
[0112] The comparative method and the method of this invention were tested under the same dataset, the same evaluation metrics, and the same experimental conditions to ensure the fairness of the experimental results. The experimental results are shown in Table 2. It can be seen that the present invention achieved a higher F1 score than the comparative method on the NASA spacecraft telemetry dataset, proving its effectiveness.
[0113] Table 2 Experimental Results
[0114]
[0115] To verify the computational efficiency advantages of this invention, the following experimental procedure was constructed for verification.
[0116] (1) Experimental data: NASA spacecraft telemetry dataset (including MSL and SMAP subsets)
[0117] (2) Comparison methods: SigLLM-D; SigLLM-P; TAMA;
[0118] (3) Evaluation metrics: To comprehensively evaluate the computational efficiency performance of this invention, this experiment uses average token consumption and average inference time as evaluation metrics. Average token consumption measures the text or multimodal input / output resources consumed by the method when calling large models or visual language models; average inference time measures the total time required to complete a single anomaly detection task. The inference time includes the complete process from the input time series, through candidate anomaly interval localization, visual language model discrimination, to the output of the final anomaly detection result.
[0119] (4) Experimental environment: All experiments were conducted on a server equipped with an NVIDIA 3090-24G GPU and CUDA version 12.1.1, and the PyTorch version used was 2.4.0. To ensure fairness, all methods were run under the same hardware environment and uniform experimental conditions.
[0120] (5) Experimental procedure:
[0121] 1. Input the time series to be detected into each comparison method and the method of the present invention;
[0122] 2. Statistically analyze the token consumption of each method during the anomaly detection task;
[0123] 3. Record the total inference time required for each method to complete anomaly detection for the entire time series;
[0124] 4. Calculate anomaly scores based on feature similarity to obtain preliminary anomaly candidate intervals;
[0125] 5. Calculate the average token consumption and average inference time for each method on the NASA spacecraft telemetry dataset;
[0126] 6. Compare and analyze the statistical results to verify the advantages of this invention in terms of computational efficiency.
[0127] The experimental results are shown in Table 3. It can be seen that the method of the present invention achieved the lowest token consumption and the shortest inference time on the NASA spacecraft telemetry dataset.
[0128] Table 3 Experimental Results
[0129]
[0130] It is understood that the time series anomaly detection method based on visual language models in the above embodiments can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the time series anomaly detection method based on visual language models provided in the above embodiments, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, they can implement the time series anomaly detection method based on visual language models as described in the above embodiments.
[0131] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the time series anomaly detection method based on a visual language model provided in the above embodiments, such as... Figure 6 As shown, it includes a memory and a processor;
[0132] The memory is used to store computer programs;
[0133] The processor is configured to implement the time series anomaly detection method based on a visual language model in the above embodiments when executing the computer program.
[0134] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.
[0135] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the time series anomaly detection method based on visual language model provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the time series anomaly detection method based on visual language model in the above embodiments.
[0136] It is understood that the computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.
[0137] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.
Claims
1. A time series anomaly detection method based on a visual language model, characterized in that, Includes the following steps: S1. Obtain the time series of the target system's operating status, perform standardization processing to obtain standardized time series data; divide the processed sequence into multiple subsequences using a sliding window, and perform time series decomposition on each subsequence to obtain multiple component sequences; convert each subsequence and its corresponding component sequences into two-dimensional line graphs, and stack them in the channel dimension to generate a multi-channel time series image; S2. Input the multi-channel temporal image into the pre-trained visual encoder to extract the basic visual feature map composed of multiple image patch features; The basic visual feature maps are pooled at different scales to obtain a multi-scale visual feature map set. S3. Based on a set of multi-scale visual feature maps, multi-scale anomaly score maps are generated through cross-window patch feature matching and similarity calculation; the multi-scale anomaly score maps are fused and mapped to the original time series to generate a one-dimensional anomaly score sequence, thereby locating the preliminary anomaly candidate intervals. S4. Convert the complete standardized time series data into a two-dimensional line graph and label the preliminary anomaly candidate intervals. Combined with the preset prompt words, input the labeled line graph and anomaly candidate interval information into the pre-trained visual language model. The visual language model will verify and correct the anomaly candidate intervals and output a formatted result containing the final anomaly interval, confidence level and anomaly description text.
2. The time series anomaly detection method based on a visual language model according to claim 1, characterized in that, The operating status time series is a monitoring data sequence from industrial equipment sensors.
3. The time series anomaly detection method based on a visual language model according to claim 1, characterized in that, In S1, time series decomposition is performed on each subsequence after division to extract trend components, seasonal components and residual components; the original data of each subsequence, as well as the trend components and seasonal components obtained from the decomposition, are mapped into two-dimensional line charts of uniform size, wherein the two-dimensional line charts do not contain coordinate axes and legend information. The three line charts are stacked along the channel dimension to generate a three-channel time series image.
4. The time series anomaly detection method based on a visual language model according to claim 1, characterized in that, The pre-trained visual encoder is a model based on the Vision Transformer architecture, and the patch-level features of the penultimate layer output of the visual encoder are extracted as the basic visual feature map.
5. The time series anomaly detection method based on a visual language model according to claim 1, characterized in that, The specific process of pooling the basic visual feature map at different scales is as follows: The basic visual feature map is subjected to average pooling operation using pooling kernels of different sizes with a pooling step size of 1. Contextual information from different spatial neighborhoods is aggregated to obtain the corresponding multi-scale visual feature map set.
6. The time series anomaly detection method based on a visual language model according to claim 1, characterized in that, In S3, the step of generating a multi-scale anomaly score map through cross-window patch feature matching and similarity calculation includes: For each patch feature in the visual feature map of the current sliding window at the current scale, calculate its cosine similarity with all patch features in the visual feature maps of all other sliding windows at the same scale. Take the maximum cosine similarity value among all the calculation results, subtract the maximum value from 1, and use the result as the anomaly score of the patch feature to obtain the anomaly score map at this scale.
7. The time series anomaly detection method based on a visual language model according to claim 6, characterized in that, In S3, the fusion of multi-scale anomaly score maps and mapping to the original time series generates a one-dimensional anomaly score sequence, thereby locating preliminary anomaly candidate intervals, including: Upsample the anomaly score maps at each scale to the same spatial resolution as the base visual feature maps; Harmonic averaging is performed on the upsampled multi-scale anomaly score maps to obtain a unified two-dimensional anomaly score map; Based on the correspondence between image columns in multi-channel time series images and time steps in the original time series, the anomaly scores of all spatial locations corresponding to each time point in the two-dimensional anomaly score image are aggregated, and a one-dimensional anomaly score sequence is generated by calculating a specified quantile. Based on a preset discrimination threshold, all consecutive time periods in which the abnormal scores exceed the threshold are extracted from the one-dimensional abnormal score sequence as preliminary abnormal candidate intervals.
8. The time series anomaly detection method based on a visual language model according to claim 1, characterized in that, In S4, the preset prompt word contains instructions for guiding the visual language model to perform the following operations: Based on the global time series trend and continuity in the input labeled two-dimensional line chart, the preliminary abnormal candidate intervals are verified and intervals that conform to the overall trend, seasonality, or normal fluctuations are eliminated. Supplement the identification of intervals that were missed due to weak visual features, but which disrupt temporal continuity, deviate from global trends, or have obvious statistical anomalies; Output the final anomaly range, corresponding confidence level, and anomaly description text in JSON format.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the time series anomaly detection method based on a visual language model as described in any one of claims 1 to 8.
10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the time series anomaly detection method based on a visual language model as described in any one of claims 1 to 8 when executing the computer program.