Multi-granularity anomaly detection system for time series data stream

Abstract

The invention belongs to the technical field of time sequence data streams, and particularly relates to a multi-granularity anomaly detection system for a time sequence data stream, which comprises a data acquisition and preprocessing module used for firstly performing multi-source data acquisition, then cleaning and standardizing the data and then generating data with different time granularities; the multi-granularity feature extraction module is used for firstly performing time sequence feature extraction, then performing window feature extraction and then performing multi-granularity feature fusion; and the anomaly detection model module is used for detecting whether the data fused by the multi-granularity feature extraction module is abnormal or not, and then improving the accuracy and stability of detection according to an integrated learning model. Compared with a traditional detection method, the method has the advantages that by arranging the anomaly detection model module, the detection accuracy and timeliness of complex mode anomalies can be greatly improved, and economic losses caused by the fact that enterprises do not find anomalies in time are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data streams, and in particular to a multi-granularity anomaly detection system for time series data streams. Background Art

[0002] In the digital age, the amount of data generated by various industries is exploding, and much of this data is continuously generated in the form of time-series data streams. Whether it's transaction data in the financial sector, sensor data in industrial production, or user behavior data in the internet industry, these time-series data streams contain rich information, reflecting the operational status of systems or businesses. However, this data often contains various anomalies, such as fraud in financial transactions, equipment failures in industrial production, and malicious attacks in internet businesses. Timely and accurate detection of these anomalies is crucial for ensuring stable system operation, mitigating risks, and improving business efficiency.

[0003] However, traditional anomaly detection methods have the following problems when facing complex and changing time series data streams:

[0004] 1. Complex pattern anomaly detection is difficult: Anomalies in time-series data streams are not simply isolated point anomalies; they often present complex sequence pattern anomalies. For example, in e-commerce sales data, product sales normally exhibit certain seasonal fluctuations over time. For example, sales of a certain brand of clothing increase in the autumn and winter, and decrease in the spring and summer. However, if, during a certain spring and summer season, the brand's sales suddenly increase significantly for several consecutive weeks, and the growth pattern differs from the growth pattern caused by previous promotional activities, this abnormal sequence pattern is difficult to capture in a timely and accurate manner by traditional detection methods. Current advanced algorithms have a low overall baseline in benchmark tests for complex pattern anomaly detection, resulting in companies being unable to detect such anomalies in a timely manner, potentially missing the optimal time to respond and causing economic losses.

[0005] 2. Long-term memory and invalid warning issues: Many existing detection algorithms have short memory cycles and lack a deep understanding of normal patterns in historical data. For example, network traffic monitoring shows a steady growth trend over a period of time, with occasional small fluctuations due to normal business activities. However, due to the algorithm's short memory cycle, when a traffic change similar to a previous normal small fluctuation occurs, the algorithm may misidentify it as an anomaly, generating an invalid warning. Invalid warnings not only consume a lot of manpower to investigate and handle, but can also trigger a "crying wolf" effect, masking the true risks. When serious system anomalies occur, relevant personnel may relax their vigilance due to the excessive number of invalid warnings and fail to take effective intervention measures in a timely manner.

[0006] 3. Insufficient adaptability to concept drift: The distribution of time-series data streams changes over time, a phenomenon known as concept drift. In industrial production, the normal distribution range of data such as temperature and pressure collected by equipment sensors may gradually shift due to factors such as adjustments to production processes and slight differences in raw materials. If the detection system cannot promptly adapt to this concept drift, persistent false alarms will occur. For example, when the equipment is operating normally, the temperature fluctuates within a fixed range. However, after adjustments to the production process, the normal temperature range increases. However, if the detection system still judges according to the old standard, a large amount of normal data will be misclassified as abnormal, seriously affecting production efficiency and the judgment of the equipment's true operating status.

[0007] 4. Multi-granularity data fusion and anomaly detection: Time series data streams have different time granularities, such as minute, hour, and day. Data of different granularities contain different levels of information. For example, minute-level data can reflect the details of instantaneous system changes, while day-level data reflects the system's long-term trends. In practical applications, anomaly detection requires the comprehensive utilization of multi-granularity data to more comprehensively and accurately detect anomalies. However, how to effectively fuse this multi-granularity data and perform accurate anomaly detection on the fused data remains an urgent problem. For example, in the power system, meter data contains both real-time minute-level electricity consumption data and hourly and daily-level statistical data. Failure to properly fuse this multi-granularity data may miss anomalies across time granularities. For example, a user's overall electricity consumption may be normal throughout the day, but unusually high peaks may occur in specific minutes. Such anomalies may not be detected by detection methods based solely on daily or minute-level data.

[0008] Based on the above, a multi-granularity anomaly detection system for time series data streams is invented. Summary of the Invention

[0009] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0010] A multi-granularity anomaly detection system for time series data streams, which includes:

[0011] The data acquisition and preprocessing module is used to collect multi-source data, clean and standardize the data, and then generate data of different time granularities;

[0012] The multi-granularity feature extraction module is used to first extract time series features, then perform window feature extraction, and then perform multi-granularity feature fusion;

[0013] The anomaly detection model module is used to first detect whether the data fused by the multi-granularity feature extraction module contains anomalies, and then improve the accuracy and stability of detection based on the integrated learning model;

[0014] The anomaly scoring and alarm module is used to first calculate the anomaly score of the output results of the anomaly detection model module, and then set the alarm threshold so that relevant personnel can be notified when the anomaly score exceeds the alarm threshold;

[0015] The abnormal cause tracing module is used to conduct in-depth analysis of detected abnormal data and use association rule mining and causal analysis techniques to explore the root causes of the abnormalities;

[0016] Intelligent prediction and prevention module, which uses time series prediction algorithms and machine learning models to predict future abnormal situations;

[0017] The model evaluation and optimization module is used to first evaluate the performance of the anomaly detection model, then monitor the performance indicators of the model in actual applications in real time, and then improve the model using corresponding optimization strategies based on the model evaluation results.

[0018] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the data acquisition and preprocessing module includes:

[0019] Multi-source data acquisition module, used to collect time series data streams from various data sources, including databases, log files, and sensors;

[0020] Data cleaning module, used to clean the collected data and remove noise data, duplicate data and missing values;

[0021] The data standardization module is used to standardize data from different sources and at different scales to make them have a unified scale, which facilitates subsequent analysis and modeling;

[0022] The multi-granularity data generation module is used to aggregate the original fine-grained data to generate data of different time granularities according to business needs.

[0023] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the multi-granularity feature extraction module includes:

[0024] A time series feature extraction module is used to extract multiple time series features based on data of different time granularities. The multiple time series features include trend features, seasonal features, and periodic features.

[0025] The window feature extraction module is used to extract local features from time series data in a sliding window manner;

[0026] The multi-granularity feature fusion module is used to fuse the features extracted from data of different granularities to form a more comprehensive feature representation.

[0027] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the anomaly detection model module includes:

[0028] The anomaly detection module uses an advanced deep learning architecture as an anomaly detection model to effectively capture the contextual dependencies of long-period sequences in time-series stream data using a self-attention mechanism, improving the ability to detect complex pattern anomalies. The model learns from normal data and constructs a pattern representation of normal data. When the data fused by the multi-granularity feature extraction module input differs significantly from the learned normal pattern, it is identified as an anomaly.

[0029] The ensemble learning model module is used to build an ensemble learning model by combining multiple different types of anomaly detection algorithms. The model can then be used to synthesize the detection results of each model through voting and weighted averaging to improve the accuracy and stability of detection.

[0030] The model adaptive adjustment module is used to update model parameters in real time by continuously receiving new data, so that the model can track changes in data distribution in a timely manner.

[0031] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the anomaly scoring and alarm module includes:

[0032] Anomaly score calculation module, used to calculate the anomaly score of each data point or data segment based on the output results of the anomaly detection model module. The anomaly score reflects the degree of deviation of the data from the normal pattern. The higher the score, the greater the degree of anomaly.

[0033] The alarm threshold setting module is used to set reasonable alarm thresholds based on business needs and historical data, so that an alarm can be triggered when the anomaly score exceeds the alarm threshold;

[0034] The alarm notification module is used to promptly notify relevant personnel through various means after an alarm is triggered, including email, text messages, and instant messaging tools.

[0035] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the anomaly cause tracing module includes:

[0036] The abnormal data location and collection module is used to accurately locate abnormal data fragments in the original time series data stream and pre-processed multi-granularity data based on the timestamp of the abnormal occurrence and data source identification information after the abnormal detection model module outputs the abnormal result. At the same time, it collects upstream and downstream data related to the abnormal data to expand the data scope and obtain more comprehensive context information;

[0037] The feature association analysis module is used to first extract features from the collected data, and then use the association rule mining algorithm to analyze the correlation between the features and find the feature combinations that are closely related to the abnormal data;

[0038] The causal relationship mining module is used to further determine the root cause of the anomaly based on the feature combination obtained through analysis and apply the causal analysis method;

[0039] The cause prioritization module is used to assign a priority score to each cause based on the degree of impact of each cause on the abnormal result and the probability of occurrence after determining multiple abnormal causes;

[0040] The traceability result output module is used to output the abnormal causes and related analysis processes in the form of a combination of visual charts and text reports.

[0041] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the intelligent prediction and prevention module includes:

[0042] The data preprocessing and feature engineering module is used to preprocess the received data after the anomaly detection model module outputs data. At the same time, it can also perform feature engineering operations based on data characteristics and business needs;

[0043] The prediction model selection and training module is used to select the corresponding prediction model from a variety of prediction algorithms based on data characteristics and prediction targets. After the prediction model is selected, the preprocessed and feature-engineered data is divided into training and test sets, and the cross-validation method is used to optimize the parameters of the prediction model.

[0044] The anomaly prediction and risk assessment module is used to use the trained prediction model to predict future data, generate a prediction sequence, and compare the prediction sequence with the pre-set normal data threshold range to determine whether there is an abnormal tendency. If the predicted value exceeds the threshold range, it is considered that an abnormal situation will occur in the future. At the same time, combining historical abnormal data and current prediction results, the risk assessment model is used to calculate the probability of anomaly occurrence and the potential impact;

[0045] The prevention strategy generation module is used to generate corresponding prevention strategies based on anomaly prediction and risk assessment results, combined with business scenarios and system rules.

[0046] As a preferred solution of the multi-granularity anomaly detection system for time series data streams described in the present invention, the model evaluation and optimization module includes:

[0047] The performance evaluation module is used to evaluate the performance of the anomaly detection model using a variety of evaluation indicators. Different evaluation indicators can reflect the performance of the model from different perspectives.

[0048] The model performance monitoring module is used to monitor the performance indicators of the anomaly detection model in actual applications in real time, so as to display the performance trend of the model through a visual interface;

[0049] The model optimization module is used to improve the model based on the model evaluation results using corresponding optimization strategies, which include adjusting model parameters, increasing training data, and improving model architecture.

[0050] Compared with existing technologies:

[0051] 1. Regarding the difficulty in detecting complex pattern anomalies: By setting up an anomaly detection model module, the present invention can greatly improve the accuracy and timeliness of detecting complex pattern anomalies compared to traditional detection methods, thereby avoiding economic losses to enterprises due to failure to detect anomalies in a timely manner.

[0052] 2. Regarding the problems of long-term memory and invalid warnings: The present invention can enhance the memory of normal patterns of historical data by integrating a learning model module and a model adaptive adjustment module, and then can make accurate judgments based on long-term memory, reduce the generation of invalid warnings, avoid the "wolf cry" effect, and ensure that real risks can receive timely attention and processing.

[0053] 3. Regarding the problem of insufficient adaptability to concept drift: The present invention can timely track changes in data distribution, avoid continuous false warnings due to concept drift, ensure production efficiency, and accurately reflect the actual operating status of the equipment.

[0054] 4. Regarding the problem of multi-granularity data fusion and anomaly detection: Through the data acquisition and preprocessing module and the multi-granularity feature extraction module, the present invention can not only comprehensively capture anomalies across time granularities, but also effectively improve the comprehensiveness and accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the framework of the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0057] The present invention provides a multi-granularity anomaly detection system for time series data streams. Figure 1 ;

[0058] The system includes: a data acquisition and preprocessing module, which is used to first collect multi-source data, then clean and standardize the data, and then generate data of different time granularities; a multi-granularity feature extraction module, which is used to first extract time series features, then perform window feature extraction, and then perform multi-granularity feature fusion; an anomaly detection model module, which is used to first detect whether there are anomalies in the data fused by the multi-granularity feature extraction module, and then improve the accuracy and stability of detection based on the integrated learning model; an anomaly scoring and alarm module, which is used to first calculate the anomaly score for the output results of the anomaly detection model module, and then set the alarm threshold so that relevant personnel can be notified when the anomaly score exceeds the alarm threshold; an anomaly cause tracing module, which is used to conduct in-depth analysis of the detected anomaly data and use association rule mining and causal analysis techniques to explore the root causes of the anomaly; an intelligent prediction and prevention module, which is used to use time series prediction algorithms and machine learning models to predict future anomalies; a model evaluation and optimization module, which is used to first evaluate the performance of the anomaly detection model, then monitor the performance indicators of the model in actual applications in real time, and then improve the model based on the model evaluation results using corresponding optimization strategies.

[0059] The data acquisition and preprocessing module includes: a multi-source data acquisition module for collecting time series data streams from various data sources, including databases, log files, and sensors; a data cleaning module for cleaning the collected data to remove noise data, duplicate data, and missing values; a data standardization module for standardizing data from different sources and different magnitudes to give them a unified scale to facilitate subsequent analysis and modeling; and a multi-granularity data generation module for aggregating original fine-grained data to generate data of different time granularities according to business needs.

[0060] The multi-granularity feature extraction module includes: a time series feature extraction module, which is used to extract multiple time series features based on data of different time granularities, and the multiple time series features include trend features, seasonal features, and periodic features; a window feature extraction module, which is used to extract local features on time series data in a sliding window manner; and a multi-granularity feature fusion module, which is used to fuse features extracted from data of different granularities to form a more comprehensive feature representation.

[0061] The anomaly detection model module includes: an anomaly detection module, which is used to adopt an advanced deep learning architecture as an anomaly detection model to effectively capture the contextual dependencies of long-period sequences in time series stream data using a self-attention mechanism, thereby improving the ability to detect complex pattern anomalies. The model constructs a pattern representation of normal data by learning normal data, so that when the data fused by the input multi-granularity feature extraction module differs significantly from the learned normal pattern, it is determined to be an anomaly; an integrated learning model module, which is used to combine multiple different types of anomaly detection algorithms to construct an integrated learning model, so as to integrate the detection results of each model through voting and weighted averaging to improve the accuracy and stability of detection; a model adaptive adjustment module, which is used to update the model parameters in real time by continuously receiving new data, so that the model can track changes in data distribution in a timely manner.

[0062] The anomaly scoring and alarm module includes: an anomaly scoring calculation module, which is used to calculate the anomaly score of each data point or data segment based on the output results of the anomaly detection model module. The anomaly score reflects the degree of deviation of the data from the normal pattern, and the higher the score, the greater the degree of anomaly; an alarm threshold setting module, which is used to set a reasonable alarm threshold based on business needs and historical data, so that an alarm can be triggered when the anomaly score exceeds the alarm threshold; an alarm notification module, which is used to promptly notify relevant personnel through various means after the alarm is triggered, including email, text message, and instant messaging tools.

[0063] The abnormal cause tracing module includes: an abnormal data positioning and collection module, which is used to accurately locate abnormal data fragments in the original time series data stream and pre-processed multi-granularity data according to the timestamp of the abnormal occurrence and the data source identification information after the abnormal result is output by the abnormal detection model module, and at the same time, collect upstream and downstream data related to the abnormal data to expand the data range in order to obtain more comprehensive context information; a feature association analysis module, which is used to first extract features from the collected data, and then use the association rule mining algorithm to analyze the correlation between each feature to find the feature combination closely related to the abnormal data; a causal relationship mining module, which is used to further determine the root cause of the abnormality by applying the causal analysis method based on the feature combination obtained by analysis; a cause priority sorting module, which is used to assign a priority score to each cause according to the degree of influence of each cause on the abnormal result and the probability factor of occurrence after determining multiple abnormal causes; a tracing result output module, which is used to output the obtained abnormal cause and related analysis process in the form of a combination of visual charts and text reports.

[0064] By setting up an abnormal cause tracing module, it can help operation and maintenance personnel quickly locate problems, take targeted solutions, effectively reduce troubleshooting time, and reduce production losses.

[0065] The intelligent prediction and prevention module includes: a data preprocessing and feature engineering module, which is used to preprocess the received data after the anomaly detection model module outputs the data, and at the same time, it can also perform feature engineering operations based on the data characteristics and business needs; a prediction model selection and training module, which is used to select a corresponding prediction model from a variety of prediction algorithms based on data characteristics and prediction targets. After the prediction model is selected, the preprocessed and feature-engineered data is divided into a training set and a test set, and the cross-validation method is used to optimize the parameters of the prediction model; an anomaly prediction and risk assessment module, which is used to use the trained prediction model to predict future data, generate a prediction sequence, and compare the prediction sequence with a pre-set normal data threshold range to determine whether there is an abnormal tendency. If the predicted value exceeds the threshold range, it is considered that an abnormal situation will occur in the future. At the same time, combined with historical abnormal data and current prediction results, the risk assessment model is used to calculate the probability of anomaly occurrence and the potential impact degree; a prevention strategy generation module, which is used to generate corresponding prevention strategies based on the anomaly prediction and risk assessment results, combined with business scenarios and system rules.

[0066] By setting up an intelligent prediction and prevention module, we can not only predict possible abnormal situations in the future, but also provide corresponding prevention strategies and suggestions for the predicted abnormalities, thereby helping enterprises to prepare in advance and reduce the probability of abnormal events.

[0067] The model evaluation and optimization module includes: a performance evaluation module for evaluating the performance of the anomaly detection model using multiple evaluation indicators. Different evaluation indicators can reflect the performance of the model from different perspectives; a model performance monitoring module for real-time monitoring of the performance indicators of the anomaly detection model in actual applications so as to display the performance change trend of the model through a visual interface; and a model optimization module for improving the model based on the model evaluation results using corresponding optimization strategies. The optimization strategies include adjusting model parameters, increasing training data, and improving the model architecture.

[0068] During specific use, the operating steps of those skilled in the art are as follows:

[0069] S1: The multi-source data acquisition module collects time series data streams from various data sources. After collection, the data cleaning module cleans the collected data to remove noise, duplicate data, and missing values. After cleaning, the data standardization module standardizes data from different sources and at different scales to achieve a unified scale, facilitating subsequent analysis and modeling. After standardization, the multi-granularity data generation module aggregates the original fine-grained data to generate data of different time granularities based on business needs.

[0070] S2: The time series feature extraction module extracts multiple time series features based on data of different time granularities. After extraction, the window feature extraction module extracts local features on the time series data using a sliding window. After extraction, the multi-granularity feature fusion module fuses the features extracted from data of different granularities to form a more comprehensive feature representation.

[0071] S3: The anomaly detection module uses an advanced deep learning architecture as an anomaly detection model to effectively capture the contextual dependencies of long-period sequences in time-series stream data using the self-attention mechanism, thereby improving the ability to detect complex pattern anomalies. The model constructs a pattern representation of normal data by learning from normal data. When the data fused by the input multi-granularity feature extraction module differs significantly from the learned normal pattern, it is judged as an anomaly. Next, an ensemble learning model is constructed by combining multiple different types of anomaly detection algorithms through the ensemble learning model module. The detection results of each model are integrated through voting and weighted averaging to improve the accuracy and stability of detection. Afterwards, the model adaptive adjustment module continuously receives new data and updates the model parameters in real time, enabling the model to track changes in data distribution in a timely manner.

[0072] S4: The anomaly score calculation module calculates the anomaly score for each data point or data segment based on the output of the anomaly detection model module. After the calculation, the alarm threshold setting module sets a reasonable alarm threshold based on business needs and historical data, so that an alarm can be triggered when the anomaly score exceeds the alarm threshold. After the setting, the alarm notification module can promptly notify relevant personnel through various means after the alarm is triggered, including email, SMS, and instant messaging tools;

[0073] S5: After the anomaly detection model module outputs the anomaly result through the anomaly data positioning and collection module, it can accurately locate the abnormal data fragment in the original time series data stream and the pre-processed multi-granularity data according to the timestamp of the anomaly occurrence and the data source identification information. At the same time, it collects upstream and downstream data related to the abnormal data to expand the data range in order to obtain more comprehensive context information. After positioning and collection, the collected data will be first extracted through the feature association analysis module. Then, the association rule mining algorithm is used to analyze the correlation between the features and find the feature combination closely related to the abnormal data. After analysis, the causal relationship mining module will use the causal analysis method based on the feature combination obtained by analysis to further determine the root cause of the anomaly. After mining, the cause priority sorting module will be used to assign a priority score to each cause according to the degree of influence of each cause on the abnormal result and the probability factor of occurrence after determining multiple abnormal causes. After sorting, the abnormal cause and related analysis process obtained will be output through the traceability result output module in the form of a combination of visual charts and text reports;

[0074] S6: After the anomaly detection model module outputs data, the data preprocessing and feature engineering module can preprocess the received data. At the same time, feature engineering operations can be performed according to data characteristics and business needs. Then, the corresponding prediction model will be selected from a variety of prediction algorithms according to data characteristics and prediction targets through the prediction model selection and training module. After the prediction model is selected, the preprocessed and feature-engineered data will be divided into training sets and test sets, and the cross-validation method will be used to optimize the parameters of the prediction model. After training, the trained prediction model will be used to predict future data through the anomaly prediction and risk assessment module to generate a prediction sequence, and the prediction sequence will be compared with the pre-set normal data threshold range to determine whether there is an abnormal tendency. If the predicted value exceeds the threshold range, it is considered that an abnormal situation will occur in the future. At the same time, the risk assessment model is used to calculate the probability of anomaly occurrence and the potential impact of the anomaly based on the historical abnormal data and the current prediction results. After the evaluation, the prevention strategy generation module will generate a corresponding prevention strategy based on the anomaly prediction and risk assessment results, combined with the business scenario and system rules;

[0075] S7: The performance of the anomaly detection model is evaluated using a variety of evaluation indicators through the performance evaluation module. Different evaluation indicators can reflect the performance of the model from different perspectives. After the evaluation, the performance indicators of the anomaly detection model in actual application will be monitored in real time through the model performance monitoring module, so that the performance change trend of the model can be displayed through the visual interface. At the same time, the model optimization module will also use corresponding optimization strategies to improve the model according to the model evaluation results.

[0076] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A multi-granularity anomaly detection system for time series data streams, characterized by: include: The data acquisition and preprocessing module is used to collect multi-source data, clean and standardize the data, and then generate data of different time granularities; The multi-granularity feature extraction module is used to first extract time series features, then perform window feature extraction, and then perform multi-granularity feature fusion; The anomaly detection model module is used to first detect whether the data fused by the multi-granularity feature extraction module contains anomalies, and then improve the accuracy and stability of detection based on the integrated learning model; The anomaly scoring and alarm module is used to first calculate the anomaly score of the output results of the anomaly detection model module, and then set the alarm threshold so that relevant personnel can be notified when the anomaly score exceeds the alarm threshold; The abnormal cause tracing module is used to conduct in-depth analysis of detected abnormal data and use association rule mining and causal analysis techniques to explore the root causes of the abnormalities; Intelligent prediction and prevention module, which uses time series prediction algorithms and machine learning models to predict future abnormal situations; The model evaluation and optimization module is used to first evaluate the performance of the anomaly detection model, then monitor the performance indicators of the model in actual applications in real time, and then improve the model using corresponding optimization strategies based on the model evaluation results.

2. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The data acquisition and preprocessing module includes: Multi-source data acquisition module, used to collect time series data streams from various data sources, including databases, log files, and sensors; Data cleaning module, used to clean the collected data and remove noise data, duplicate data and missing values; The data standardization module is used to standardize data from different sources and at different scales to make them have a unified scale, which facilitates subsequent analysis and modeling; The multi-granularity data generation module is used to aggregate the original fine-grained data to generate data of different time granularities according to business needs.

3. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The multi-granularity feature extraction module includes: A time series feature extraction module is used to extract multiple time series features based on data of different time granularities. The multiple time series features include trend features, seasonal features, and periodic features. The window feature extraction module is used to extract local features from time series data in a sliding window manner; The multi-granularity feature fusion module is used to fuse the features extracted from data of different granularities to form a more comprehensive feature representation.

4. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The anomaly detection model module includes: The anomaly detection module uses an advanced deep learning architecture as an anomaly detection model to effectively capture the contextual dependencies of long-period sequences in time-series stream data using a self-attention mechanism, improving the ability to detect complex pattern anomalies. The model learns from normal data and constructs a pattern representation of normal data. When the data fused by the multi-granularity feature extraction module input differs significantly from the learned normal pattern, it is identified as an anomaly. The ensemble learning model module is used to build an ensemble learning model by combining multiple different types of anomaly detection algorithms. The model can then be used to synthesize the detection results of each model through voting and weighted averaging to improve the accuracy and stability of detection. The model adaptive adjustment module is used to update model parameters in real time by continuously receiving new data, so that the model can track changes in data distribution in a timely manner.

5. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The abnormality scoring and alarm module includes: Anomaly score calculation module, used to calculate the anomaly score of each data point or data segment based on the output results of the anomaly detection model module. The anomaly score reflects the degree of deviation of the data from the normal pattern. The higher the score, the greater the degree of anomaly. The alarm threshold setting module is used to set reasonable alarm thresholds based on business needs and historical data, so that an alarm can be triggered when the anomaly score exceeds the alarm threshold; The alarm notification module is used to promptly notify relevant personnel through various means after an alarm is triggered, including email, text messages, and instant messaging tools.

6. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The abnormal cause tracing module includes: The abnormal data location and collection module is used to accurately locate abnormal data fragments in the original time series data stream and pre-processed multi-granularity data based on the timestamp of the abnormal occurrence and data source identification information after the abnormal detection model module outputs the abnormal result. At the same time, it collects upstream and downstream data related to the abnormal data to expand the data scope and obtain more comprehensive context information; The feature association analysis module is used to first extract features from the collected data, and then use the association rule mining algorithm to analyze the correlation between the features and find the feature combinations that are closely related to the abnormal data; The causal relationship mining module is used to further determine the root cause of the anomaly based on the feature combination obtained through analysis and apply the causal analysis method; The cause prioritization module is used to assign a priority score to each cause based on the degree of impact of each cause on the abnormal result and the probability of occurrence after determining multiple abnormal causes; The traceability result output module is used to output the abnormal causes and related analysis processes in the form of a combination of visual charts and text reports.

7. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The intelligent prediction and prevention module includes: The data preprocessing and feature engineering module is used to preprocess the received data after the anomaly detection model module outputs data. At the same time, it can also perform feature engineering operations based on data characteristics and business needs; The prediction model selection and training module is used to select the corresponding prediction model from a variety of prediction algorithms based on data characteristics and prediction targets. After the prediction model is selected, the preprocessed and feature-engineered data is divided into training and test sets, and the cross-validation method is used to optimize the parameters of the prediction model. The anomaly prediction and risk assessment module is used to use the trained prediction model to predict future data, generate a prediction sequence, and compare the prediction sequence with the pre-set normal data threshold range to determine whether there is an abnormal tendency. If the predicted value exceeds the threshold range, it is considered that an abnormal situation will occur in the future. At the same time, combining historical abnormal data and current prediction results, the risk assessment model is used to calculate the probability of anomaly occurrence and the potential impact; The prevention strategy generation module is used to generate corresponding prevention strategies based on anomaly prediction and risk assessment results, combined with business scenarios and system rules.

8. The multi-granularity anomaly detection system for time series data streams according to claim 1, characterized in that: The model evaluation and optimization module includes: The performance evaluation module is used to evaluate the performance of the anomaly detection model using a variety of evaluation indicators. Different evaluation indicators can reflect the performance of the model from different perspectives. The model performance monitoring module is used to monitor the performance indicators of the anomaly detection model in actual applications in real time, so as to display the performance trend of the model through a visual interface; The model optimization module is used to improve the model based on the model evaluation results using corresponding optimization strategies, which include adjusting model parameters, increasing training data, and improving model architecture.

Images