Time domain early warning method and system for high-frequency faults in data centers based on TS-Decomposition
Through the TS-Decomposition method, the influencing factors of timing data are decomposed, the prediction model is established, and the high-frequency fault time domain of data centers is predicted, which solves the problems of improper allocation of operation and maintenance resources and delay in fault processing in the existing technology, and realizes efficient fault warning and Resource optimization.
Patent Information
- Application Number
- CN202111255316.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-10-27
AI Technical Summary
The prior art is difficult to effectively predict the time domain of high-frequency failure in data centers, resulting in improper allocation of operation and maintenance resources and delay in troubleshooting.
Using the TS-Decomposition-based method, by obtaining and cleaning timing monitoring data, decomposing numerical influencing factors, establishing a timing prediction model, predicting future trends, and combining historical and future data to perform mixed calculations to judge the fault time domain, and realize high-frequency faults Early warning.
It improves operation and maintenance efficiency, can predict failure trends before failure occurs, reasonably allocate operation and maintenance resources, and improves the operation and maintenance capabilities of the data center.
Smart Images

Figure CN113986704B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer intelligent operation and maintenance, and specifically relates to a time domain early warning method and system for high-frequency faults in a data center based on TS-Decomposition (Time Sequence Decomposition). Background Art
[0002] In recent years, with the rapid development of 5G technology, cloud computing, the Internet of Things and other fields have achieved unprecedented development. Data centers are responsible for managing, organizing and analyzing various cloud computing resources, terminal resources, and Internet of Things device resources, ensuring the smooth operation of thousands of devices; with the explosive growth of data and the increase of computing nodes, traditional data center operation and maintenance methods have been difficult to meet the current operation and maintenance needs. At present, data centers in enterprises or universities usually run tens of thousands of data nodes, computing resources, sensors, terminals, etc., and relevant monitoring data is continuously generated every moment. How to extract effective information from massive operation and maintenance monitoring data, combine specific machine learning algorithms, and conduct data mining, trend prediction, fault alarm, etc. for monitoring indicators has become an important topic of intelligent operation and maintenance of data centers.
[0003] Among the massive operation and maintenance data, time series data has a certain degree of autocorrelation and periodicity, that is, the operation and maintenance equipment corresponding to the time series data often has a tendency to fail before the failure occurs. Taking the CPU temperature as an example, in a sustained period of time, before the CPU temperature reaches the damage value, it often maintains a high temperature threshold for a long time. For external devices, the CPU temperature is also affected to a certain extent by periodic factors such as day and night alternation and seasons. The influence of multiple factors will be reflected in the numerical changes of the time series data, which can be used as the basis for operation and maintenance analysis.
[0004] In the traditional operation and maintenance process, operation and maintenance fault handling is usually done after the fault occurs, and professional operation and maintenance personnel will locate the fault based on the fault information, and then investigate and solve it one by one. At this stage, cloud computing resources, terminals, and sensors are often not independent of each other. The correlation between faults, the large amount of fault alarm information, and the non-real-time nature of alarms all lead to difficulties in data center operation and maintenance. How to use existing machine learning algorithms to mine fault features in time series data, predict the trend of time series data, predict the high-frequency fault time domain in advance through a specific indicator library, and reasonably allocate operation and maintenance personnel and resources are important issues that need to be solved in the current intelligent operation and maintenance of data centers. Summary of the invention
[0005] In view of this, the present invention proposes a data center high-frequency fault time domain warning method and system based on TS-Decomposition, which is used to solve the problem that operation and maintenance in existing data cannot effectively predict the high-frequency fault time domain.
[0006] In a first aspect, the present invention discloses a time domain early warning method for high-frequency faults in a data center based on TS-Decomposition, the method comprising:
[0007] Obtain time series monitoring data in data center scenarios and perform data cleansing to obtain monitoring values of historical time series data;
[0008] Analyze the numerical influencing factors of time series data, decompose the time series data based on the TS-Decomposition algorithm, and calculate the numerical influencing factors based on the monitoring values of historical time series data;
[0009] Establish a time series prediction model based on the calculation results of numerical influencing factors;
[0010] According to the time series prediction model, future trend prediction is performed to obtain the predicted value of future time series data;
[0011] For different application scenarios, hybrid calculations are performed based on the monitoring values of historical time series data and the predicted values of future time series data to determine the fault time domain of the time series data;
[0012] Provide time domain warning for high-frequency faults in data centers based on the fault time domain of time series data.
[0013] Preferably, the acquiring of time series monitoring data in a data center scenario and performing data cleaning includes:
[0014] Establish a time series monitoring data indicator library in the data center scenario, filter out periodic or interval numerical time series data from the time series monitoring data in the data center scenario based on the time series monitoring data indicator library, and establish a time series data warehouse;
[0015] Perform data cleaning on the time series data in the time series data warehouse. Data cleaning includes but is not limited to outlier removal, null value filling, and data interpolation and smoothing.
[0016] Preferably, analyzing the numerical influencing factors of the time series data, performing time series decomposition on the time series data based on the TS-Decomposition algorithm, and calculating the numerical influencing factors according to the monitoring values of the historical time series data specifically includes:
[0017] The types of historical time series data after data cleaning include periodic time series data and interval time series data; for periodic time series data, its numerical influencing factors include: long-term trend factor T t , Cyclic change factor C tand irregular variation factor I t ,Based on the TS-Decomposition algorithm, the multiplication model decomposition of time series data is:
[0018] X t =T t ×C t ×I t
[0019] Among them, X t is the total change of the time series, representing the value of the known time series data itself; T t is the long-term trend factor, C t is the cyclical change factor, I t Irregular change factors;
[0020] For interval time series data, the factors affecting its value include: long-term trend factors, irregular change factors;
[0021] The time series decomposition based on the TS-Decomposition algorithm and the calculation of numerical influencing factors according to the monitoring values of the historical time series data specifically include:
[0022] Preprocess the historical time series data according to the type of time series data to eliminate the cyclic change factor C t Impact on the values of historical time series data;
[0023] The trend function is fitted using the trend inference method, and the long-term trend factor T is obtained by analysis. t , based on the preprocessing results and the obtained long-term trend factors, the irregular change factor I is calculated t , where T t With X t have the same dimension, C t with I t is a ratio.
[0024] Preferably, the preprocessing according to the type of historical time series data to eliminate the influence of cyclic variation factors on the time series data values specifically includes:
[0025] For periodic time series data, sliding average is performed to eliminate the influence of cyclical change factors on the value of time series data. The formula for sliding average is:
[0026]
[0027] Among them, X t is the known total change of the time series, representing the value of the time series itself; C t is the cyclic change; D' is the sliding average processing result of the periodic time series data;
[0028] For interval trend time series data, the interval trend time series data is regarded as a small range of periodic data, which does not contain cyclical change factors and does not need to be eliminated;
[0029] The time series data after preprocessing to eliminate cyclic variation factors is recorded as the moving average series D.
[0030] Preferably, the trend function is fitted by the trend inference method, and the long-term trend factors are analyzed and obtained. According to the preprocessing results and the obtained long-term trend factors, the irregular change factors are calculated and specifically include:
[0031] By observing and reasoning, analyzing the trend of the moving average series D obtained by preprocessing, a trend prediction model is selected; the linear trend model includes a linear trend prediction model and an exponential trend prediction model;
[0032] Based on the moving average series D obtained by preprocessing, the least squares method is used to fit the parameters of the trend prediction model to obtain the trend prediction function, which is used as the long-term trend factor T t ;
[0033] Based on the moving average series D and the calculated long-term trend factor T t Calculating the Stochastic Factor
[0034] Preferably, the step of establishing a time series prediction model according to the calculation results of the numerical influencing factors, and performing future trend prediction according to the time series prediction model to obtain the predicted value of future time series data specifically includes:
[0035] According to the long-term trend factor T of the historical time series data calculated t , Cyclic change factor C t and irregular variation factor I t Estimate the long-term trend factor T of the time series t+1 in the same period / same interval in the future t+1 , Cyclic change factor C t+1 and irregular variation factor I t+1 ;
[0036] Long-term trend factor T in the time series of the same period / same interval in the future t+1 , Cyclic change factor C t+1 and irregular variation factor I t+1 Get the time series prediction model:
[0037] X t+1 =T t+1 ×C t+1 ×I t+1
[0038] The predicted value X of future time series data is obtained by predicting the time series prediction model t+1 .
[0039] Preferably, for different application scenarios, hybrid calculation is performed based on the monitoring value of historical time series data and the predicted value of future time series data to determine the fault time domain of the time series data specifically including:
[0040] The monitoring values of historical time series data are recorded as the sequence A{a1,a2,a3,...,an}, and the predicted values of future time series data are recorded as the sequence B{b1,b2,b3,...,bn}. According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on the sequence A and the sequence B to obtain the hybrid sequence C. n , the mixed sequence C n Compare with the lower threshold value and the upper threshold value of the corresponding indicator in different scenarios, and obtain the fault time domain of the time series data according to the comparison result;
[0041] According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on sequence A and sequence B to obtain a hybrid sequence C. n , specifically including:
[0042] For periodic time series data whose value fluctuation is less than the first preset threshold, the superposition method is used, and the mixed sequence is recorded as sequence C n ,have
[0043] C n =A n1 +A n2 +...+A nk +B n
[0044] Where k is the number of small periods of periodic data, n is the number of values contained in one period, and A nk represents the monitoring value of the historical time series data of the kth small period, B n Represents the predicted value of future time series data;
[0045] For periodic time series data whose value fluctuation is greater than the second preset threshold, the mean method is used, and the mixed sequence is recorded as sequence C n ,have
[0046]
[0047] A nm Indicates the monitoring value of the historical time series data of the mth small period;
[0048] For time-sensitive periodic time series data, a weighted method is used, and the mixed sequence is recorded as sequence C n ,have:
[0049]
[0050] Where k is the number of small periods of periodic data, n is the number of values contained in a period, and λ is the weighting index, which ranges from [0,1];
[0051] For periodic time series data with a stable trend, the peak method is used and the mixed sequence is recorded as sequence C n ,have
[0052] C n =max(A n1 ||A n2 ...||A nk ||B n )
[0053] The resulting sequence C n It is a sequence consisting of the maximum values of single time nodes in each small period.
[0054] In a second aspect, the present invention discloses a data center high-frequency fault time domain early warning system based on TS-Decomposition, the method comprising:
[0055] Data cleaning module: obtains time series monitoring data in the data center scenario and performs data screening and data cleaning to obtain the monitoring values of historical time series data;
[0056] Data processing module: Analyze the numerical influencing factors of historical time series data, decompose the historical time series data based on the TS-Decomposition algorithm, and calculate the numerical influencing factors based on the monitoring values of the historical time series data;
[0057] Trend prediction module: establish a trend prediction model based on the calculation results of numerical influencing factors; perform future trend prediction based on the trend prediction model to obtain the predicted value of future time series data;
[0058] Time domain judgment module: for different application scenarios, it performs mixed calculations based on the monitoring values of historical time series data and the predicted values of future time series data to judge the fault time domain of the time series data;
[0059] Fault warning module: Provides time domain warning for high-frequency faults in the data center based on the fault time domain of time series data.
[0060] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory and communication interface communicate with each other via the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method described in the first aspect of the present invention.
[0061] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method described in the first aspect of the present invention.
[0062] Compared with the prior art, the present invention has the following beneficial effects:
[0063] 1) The present invention relies on time series data monitoring in data center scenarios, extracts historical time series data with periodic or interval rows from the data center operation and maintenance data warehouse as raw data and performs data cleaning, analyzes the numerical influencing factors of the historical time series data, performs time series decomposition on the historical time series data based on the time series decomposition algorithm, and calculates the numerical influencing factors according to the monitoring values of the historical time series data; in the process of calculating the influencing factors, trend reasoning is performed to obtain a long-term trend prediction model, and then a time series prediction model is established based on the calculation results of the numerical influencing factors and future trend prediction is performed to obtain the predicted value of future time series data. The present invention mines the correlation in the time series data, analyzes the high-frequency distribution interval of faults in the data, predicts the trend of failure before the failure occurs, and improves the operation and maintenance efficiency.
[0064] 2) The present invention performs mixed calculations for different time series numerical types based on the monitored values of historical time series data and the predicted values of future time series data, determines the fault time domain of the time series data, and implements high-frequency fault time domain analysis of data center time series data under different application scenarios, providing a reference for operation and maintenance resource allocation, and can combine the operation and maintenance indicator library with the alarm distribution platform to realize high-frequency fault time domain early warning. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0066] Figure 1 It is a flow chart of the time domain early warning method for high-frequency faults in a data center based on TS-Decomposition of the present invention;
[0067] Figure 2 It is a flow chart of time series data trend prediction of the present invention;
[0068] Figure 3 is a trend reasoning flow chart of the present invention;
[0069] Figure 4 It is a flow chart of the method for acquiring high-frequency fault time domain data of the present invention;
[0070] Figure 5 This is a time domain example diagram of a high-frequency fault under the flow monitoring index of an embodiment of the present invention.
[0071] Figure 6 It is a schematic diagram of the high-frequency fault time domain early warning distribution of the present invention. DETAILED DESCRIPTION
[0072] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0073] See also Figure 1 In order to mine the correlation in time series data and analyze the high-frequency fault time domain (high-frequency fault distribution interval) in the data, the present invention proposes a data center high-frequency fault time domain early warning method based on TS-Decomposition, the method comprising:
[0074] S1. Obtain the time series monitoring data in the data center scenario and perform data screening and data cleaning to obtain the monitoring value of the historical time series data;
[0075] The time series monitoring data of the data center contains tens of thousands of operation and maintenance data information, and the data dimensions contained are also extremely complex. The present invention establishes a time series monitoring data indicator library under the data center scenario, combines the ETL (Extract-Transform-Load) technology, and filters out periodic or interval numerical time series data from the time series monitoring data under the data center scenario according to the time series monitoring data indicator library to establish a new time series data warehouse.
[0076] Specifically, the hardware equipment of the data center mainly includes computers, sensors, and terminal devices, among which the categories of computers are mainly computing nodes and storage nodes; the sensor equipment mainly includes environmental temperature sensors, humidity sensors, and lightning sensors. A series of operation and maintenance-related monitoring and service software are deployed on the hardware equipment of the data center, including but not limited to middleware such as SuperVisor, Redis, Nginx, Prometheus, Elasticsearch, and Kibana. The computer hardware layer monitoring data mainly includes hardware monitoring information of computing nodes such as CPU temperature, CPU occupancy, CPU frequency, memory usage, disk I / 0, disk usage, and disk usage. The sensor layer mainly includes monitoring information such as the ambient temperature and humidity around the sensor. The computer software layer mainly includes execution information of different software, such as Redis write rate, MySQL space occupancy, Kibana concurrent access volume, and SuperVisor service restart times. A time series monitoring data indicator library is established through the above indicators. Based on the above time series monitoring data indicator library, the time series data indicators that can be further analyzed are screened out. The time series data indicators available for use must meet three conditions:
[0077] (1) Time continuity. The data must be continuous in the time dimension and cannot be a software service that restarts intermittently, such as the Redis write rate.
[0078] (2) Numeric type. The data must be a monitorable numeric type and cannot be a Boolean value or status value.
[0079] (3) Autocorrelation. The changes in data over time must have a certain degree of autocorrelation, and numerical anomalies caused by scheduled tasks or human operations must be eliminated.
[0080] The screened time series data indicators are cleaned, and methods such as outlier removal, null value filling, and data interpolation and smoothing are used to form a custom data center time series data warehouse, which is used to store the monitoring values of the cleaned historical time series data as input data for subsequent steps.
[0081] S2. Analyze the numerical influencing factors of time series data, decompose the time series data based on the TS-Decomposition algorithm, and calculate the numerical influencing factors based on the monitoring values of historical time series data;
[0082] See also Figure 2The time series data trend prediction flow chart extracts the time series data to be analyzed from the custom data center time series data warehouse, which is recorded as series A. The cleaned historical time series data is generally divided into two types, namely periodic numerical time series data and interval numerical time series data without periodicity. Parameters are set, including start time, end time, time interval, and number of prediction steps. Then, the time series data is decomposed based on the TS-Decomposition algorithm, and the trend of the time series data is predicted in combination with trend reasoning to obtain the predicted value of future time series data. Specifically, step S2 includes the following sub-steps:
[0083] S21. Analyze and determine the numerical influencing factors of time series data;
[0084] The historical time series data includes periodic time series data and interval time series data. For periodic time series data, its numerical influencing factors include: long-term trend factor T t , Cyclic change factor C t and irregular variation factor I t , based on the TS-Decomposition algorithm, the multiplication model decomposition of time series data is: X t =T t ×C t ×I t
[0085] Among them, X t is the total change of the time series, representing the value of the known time series data itself; T t is the long-term trend factor, C t is the cyclical variable factor, I t Irregular change factors;
[0086] For interval time series data, the factors affecting its values include: long-term trend factors and irregular change factors.
[0087] After analyzing and determining the numerical influencing factors of time series data, the time series is decomposed based on the TS-Decomposition algorithm, and then the numerical influencing factors are calculated based on the monitoring values of historical time series data, including:
[0088] S22, preprocessing the historical time series data according to the type of the time series data to eliminate the influence of cyclical change factors on the values of the historical time series data;
[0089] Specifically, a sliding average is performed on the periodic time series data to eliminate the influence of cyclical variation factors on the time series data values. The sliding average formula is:
[0090]
[0091] Among them, X tis the known total change of the time series, representing the value of the time series itself; C t is the cyclic change; D' is the sliding average processing result of the periodic time series data;
[0092] For interval trend time series data, the interval trend time series data is regarded as a small range of periodic data, which does not contain cyclical variation factors and does not need to be eliminated. Since the data obtained before has no periodicity and the random factors are also very small, the data can be considered to have been smoothed.
[0093] The historical time series data after preprocessing to eliminate cyclical change factors is recorded as the moving average series D, then D = T t ×I t .
[0094] S23, use the trend inference method to fit the trend function and analyze the long-term trend factor T t , based on the preprocessing results and the obtained long-term trend factors, the irregular change factor I is calculated t Among them, T t With X t have the same dimension, C t with I t is a ratio.
[0095] Given that D = T t ×I t The existing series already contains a set of irregular change factors and long-term trend factors. First, the long-term trend factor T is obtained through trend reasoning and analysis. t . Trend inference methods include linear trend extension, curve trend method, function model inference method (exponential, growth curve, envelope curve), etc.
[0096] For details, please refer to Figure 3 The trend reasoning flow chart first analyzes the trend of the moving average series D obtained by preprocessing through observation and reasoning, and selects a trend prediction model; the linear trend model includes a linear trend prediction model T t =ax+b, exponential trend prediction model Where x represents a time point in a time series t, and can also be a prediction model such as quadratic fitting, curve fitting, and multivariate fitting.
[0097] Then, data is collected, and the moving average series D obtained by the above preprocessing is the data required for trend reasoning of the present invention.
[0098] The least squares method is used to fit the parameters of the trend prediction model and obtain the trend prediction function, which is used as the long-term trend factor T. tThe trend prediction function obtained can be used to perform trend analysis and reasoning, such as extrapolating trends based on historical trends to obtain the trends of time series in the same cycle / same interval in the future.
[0099] Based on the moving average series D and the calculated long-term trend factor T t Calculating Stochastic Volatility Factors
[0100] S3. Establish a time series prediction model based on the calculation results of the numerical influencing factors, and predict future trends based on the time series prediction model to obtain the predicted value of future time series data;
[0101] According to the long-term trend factor T of the historical time series data calculated t , Cyclic change factor C t and irregular variation factor I t Perform trend extrapolation to obtain the long-term trend factor T under the same period / same interval time series t+1 in the future t+1 , Cyclic change factor C t+1 and irregular variation factor I t+1 ; Generally speaking, the long-term trend factor T t In the future, the same period / same interval of the time series is used to predict the function in the trend prediction model. The cyclic change factor C t+1 and irregular variation factor I t+1 Repeat changes in the same cycle / same interval time series in the future.
[0102] According to the long-term trend factor T in the time series of the same period / same interval in the future t+1 , Cyclic change factor C t+1 and irregular variation factor I t+1 Get the time series prediction model:
[0103] X t+1 =T t+1 ×C t+1 ×I t+1
[0104] The predicted value X of future time series data is obtained by predicting the time series prediction model t+1 , recorded as sequence B.
[0105] S4. For different time series value types, a mixed calculation is performed based on the monitoring values of historical time series data and the predicted values of future time series data to determine the fault time domain of the time series data;
[0106] Specifically, the monitoring values of historical time series data are recorded as the sequence A{a1,a2,a3,...,an}, and the predicted values of future time series data are recorded as the sequence B{b1,b2,b3,...,bn}. According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on the sequence A and the sequence B to obtain the hybrid sequence C. n , the mixed sequence C n Compare with the lower threshold value and the upper threshold value of the corresponding indicator in different scenarios, and obtain the fault time domain of the time series data according to the comparison result;
[0107] See also Figure 4 The flow chart of the high-frequency fault time domain data acquisition method is as follows. According to different time series numerical types, different hybrid strategies are used to perform hybrid calculations on sequence A and sequence B to obtain the hybrid sequence C. n , specifically including:
[0108] (1) For periodic time series data whose value fluctuation is less than the first preset threshold, the value fluctuation is small, and the superposition method is used. The mixed sequence is recorded as sequence C n ,have:
[0109] C n =A n1 +A n2 +...+A nk +B n
[0110] Where k is the number of small periods of periodic data, n is the number of values contained in one period, and A nk represents the monitoring value of the historical time series data of the kth small period, B n Represents the predicted value of future time series data.
[0111] (2) For periodic time series data whose value fluctuation is greater than the second preset threshold, the value fluctuation is large, and the mean method is used. The mixed sequence is recorded as sequence C n ,have:
[0112]
[0113] A nm represents the monitoring value of the mth periodic time series data, B n Represents the predicted value of future time series data.
[0114] (3) For time-sensitive periodic time series data, such as hard disk capacity and other data that maintain a certain trend over time or network traffic and other operation and maintenance data that changes rapidly over time, a weighted method is used, and the mixed sequence is recorded as sequence C n ,have:
[0115]
[0116] Where k is the number of small periods of periodic data, n is the number of values contained in a period, and λ is the weighting index, which ranges from [0,1];
[0117] (4) For periodic time series data whose value fluctuation range is within the preset interval, the value has a stable trend. The peak value method is used and the mixed sequence is recorded as sequence C n ,have:
[0118] C n =max(A n1 ||A n2 ...||A nk ||B n )
[0119] The resulting sequence C n It is a sequence consisting of the maximum values of single time nodes in each small period.
[0120] Figure 5 This is a time domain example diagram of a high-frequency fault under the flow monitoring index of a certain embodiment.
[0121] S5. Perform time domain warning of high-frequency faults in the data center based on the fault time domain of the time series data.
[0122] The present invention is based on data center operation and maintenance fault indicators and user-defined fault analysis indicators, relies on the data center alarm distribution infrastructure, combines operation and maintenance resource allocation with actual scenarios, and realizes time domain early warning of high-frequency faults in the data center.
[0123] Relying on the custom data center time series data warehouse, the corresponding custom high-frequency fault time domain data warehouse is designed. The data warehouse needs to meet the following characteristics: consistent with the custom data center time series data warehouse, that is, the time series contained are consistent, the time series intervals are consistent, the numerical units are consistent, and the number of data nodes is consistent. Each extracted time series content corresponds to a high-frequency fault time domain interval series. The only difference between the two is that the high-frequency fault time domain data warehouse data is calculated based on the data of the custom data center time series data warehouse.
[0124] See also Figure 6 , the high-frequency fault time domain warning distribution diagram of the present invention, designs the high-frequency fault time domain warning database of the data center time series data, and after obtaining the high-frequency fault time domain, the monitoring service software monitors the warning database information in real time, captures the abnormal time domain value, and combines the data center's own operation and maintenance warning system to realize high-frequency fault time domain warning. The fault analysis indicators in the operation and maintenance monitoring indicator library mainly include two categories:
[0125] 1. Basic time series data indicators, such as CPU utilization, CPU core occupancy, etc.
[0126] 2. Custom indicators: Alarm indicators combined with business scenarios, such as issuing an alarm when the hard disk occupancy rate reaches 70%.
[0127] Monitoring service software refers to a series of service software that combines operation and maintenance monitoring indicators and scans the data center's high-frequency fault time domain warning database in real time. After obtaining the monitoring indicators, the service software will scan the warning database through a scheduled task. Once abnormal time domain data is detected, it will pull the abnormal value record and the corresponding time interval record and distribute it to the data center operation and maintenance alarm system. The data center operation and maintenance warning system is the data center's own warning service. After receiving the fault warning, it will send a reminder to the corresponding operation and maintenance personnel like the basic operation and maintenance fault. The operation and maintenance personnel can allocate operation and maintenance resources according to the high-frequency fault time domain warning information, arrange more operation and maintenance personnel in the high-frequency fault time domain, check possible operation and maintenance faults in advance, and save operation and maintenance resources in the low-frequency fault time domain, so as to achieve the purpose of saving operation and maintenance costs.
[0128] Corresponding to the above method embodiment, the present invention also proposes a data center high-frequency fault time domain early warning system based on TS-Decomposition, the system comprising:
[0129] Data cleaning module: obtains time series monitoring data in the data center scenario and performs data screening and data cleaning to obtain the monitoring values of historical time series data;
[0130] Data processing module: Analyze the numerical influencing factors of historical time series data, decompose the historical time series data based on the TS-Decomposition algorithm, and calculate the numerical influencing factors based on the monitoring values of the historical time series data;
[0131] Trend prediction module: establish a time series prediction model based on the calculation results of numerical influencing factors; predict future trends based on the time series prediction model to obtain the predicted value of future time series data;
[0132] Time domain judgment module: for different time series value types, it performs mixed calculations based on the monitoring values of historical time series data and the predicted values of future time series data to judge the fault time domain of the time series data;
[0133] Fault warning module: Provides time domain warning for high-frequency faults in the data center based on the fault time domain of time series data.
[0134] The above method embodiments and system embodiments correspond to each other, and for a brief description of the system embodiments, please refer to the method embodiments.
[0135] The present invention mines the association relationship in the time series data through the time series decomposition algorithm, predicts the future trend of the time series data, and performs mixed calculations based on the monitoring values of the historical time series data and the predicted values of the future time series data for different time series value types, determines the fault time domain of the time series data, analyzes the high-frequency distribution interval of the faults in the data, predicts the trend of the fault before the fault occurs, and improves the operation and maintenance efficiency. The present invention realizes the high-frequency fault time domain analysis of the time series data of the data center in different application scenarios, provides a reference for the allocation of operation and maintenance resources, and can combine the operation and maintenance indicator library with the alarm distribution platform to realize the high-frequency fault time domain warning.
[0136] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.
[0137] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc., which can store program codes.
[0138] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be distributed to multiple network units. A person skilled in the art may select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment without creative effort.
[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A time domain early warning method for high-frequency faults in a data center based on TS-Decomposition, characterized in that: The method comprises: Obtain time series monitoring data in data center scenarios and perform data cleansing to obtain monitoring values of historical time series data; Analyze the numerical influencing factors of time series data, decompose the time series data based on the TS-Decomposition algorithm, and calculate the numerical influencing factors based on the monitoring values of historical time series data; A time series prediction model is established based on the calculation results of numerical influencing factors, and the trend of time series data is predicted based on the time series prediction model to obtain the predicted value of future time series data; For different time series value types, hybrid calculation is performed based on the monitoring values of historical time series data and the predicted values of future time series data to determine the fault time domain of the time series data; Perform time domain warning for high-frequency faults in data centers based on the fault time domain of time series data; The analysis of the numerical influencing factors of the time series data, the time series decomposition of the time series data based on the TS-Decomposition algorithm, and the calculation of the numerical influencing factors according to the monitoring values of the historical time series data specifically include: The historical time series data after data cleaning includes periodic time series data and interval time series data. For periodic time series data, its numerical influencing factors include: long-term trend factor T t , Cyclic change factor C t and irregular variation factor I t ,Based on the TS-Decomposition algorithm, the multiplication model decomposition of time series data is: X t =T t ×C t ×I t Among them, X t is the total change of the time series, representing the value of the known time series data itself; T t is the long-term trend factor, C t is the cyclical variable factor, I t Irregular change factors; For interval time series data, the factors affecting its value include: long-term trend factors, irregular change factors; Preprocess the historical time series data according to the type of time series data to eliminate the cyclic change factor C t Impact on the numerical values of historical time series data; The trend function is fitted using the trend inference method to obtain the long-term trend factor T t , based on the preprocessing results and the obtained long-term trend factors, the irregular change factor I is calculated t , where T t With X t have the same dimension, C t with I t is a ratio; For different time series value types, hybrid calculation is performed based on the monitoring values of historical time series data and the predicted values of future time series data to determine the fault time domain of the time series data, specifically including: The monitoring values of historical time series data are recorded as the sequence A{a1,a2,a3,...,an}, and the predicted values of future time series data are recorded as the sequence B{b1,b2,b3,...,bn}. According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on the sequence A and the sequence B to obtain the hybrid sequence C. n , the mixed sequence C n Compare with the lower threshold value and the upper threshold value of the corresponding indicator in different scenarios, and obtain the fault time domain of the time series data according to the comparison result; According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on sequence A and sequence B to obtain a hybrid sequence C. n , specifically including: 1) For periodic time series data whose value fluctuation is less than the first preset threshold, the superposition method is used, and the mixed sequence is recorded as sequence C n ,have C n =A n1 +A n2 +...+A nk +B n Where k is the number of small periods of periodic data, and n is the number of values contained in one period; A nk represents the monitoring value of the historical time series data of the kth small period, B n Represents the predicted value of future time series data; 2) For periodic time series data whose value fluctuation is greater than the second preset threshold, the mean method is used, and the mixed sequence is recorded as sequence C n : A nm Represents the monitoring value of the mth periodic time series data; 3) For time-sensitive periodic time series data, a weighted method is used, and the mixed sequence is recorded as sequence C n : Where k is the number of small periods of periodic data, n is the number of values contained in a period, and λ is the weighting index, which ranges from [0,1]; 4) For periodic time series data whose value fluctuation range is within the preset interval, the peak method is used and the mixed sequence is recorded as sequence C n : C n =max(A n1 ||A n2 ...||A nk ||B n ) The resulting sequence C n It is a sequence consisting of the maximum values of single time nodes in each small period.
2. According to claim 1, the method for early warning of high-frequency faults in a data center based on TS-Decomposition is characterized in that: The obtaining of time series monitoring data in a data center scenario and data cleaning includes: Establish a time series monitoring data indicator library in the data center scenario, filter out periodic or interval numerical time series data from the time series monitoring data in the data center scenario based on the time series monitoring data indicator library, and establish a time series data warehouse; Perform data cleaning on the historical time series data in the time series data warehouse. Data cleaning includes but is not limited to outlier removal, null value filling, and data interpolation and smoothing.
3. According to claim 1, the method for early warning of high-frequency faults in a data center based on TS-Decomposition is characterized in that: The preprocessing according to the type of historical time series data to eliminate the influence of cyclic variation factors on the time series data value specifically includes: For periodic time series data, sliding average is performed to eliminate the influence of cyclical change factors on the value of time series data. The formula for sliding average is: Among them, X t is the known total change of the time series, representing the value of the time series itself; C t is the cyclic change; D' is the sliding average processing result of the periodic time series data; For interval trend time series data, the interval trend time series data is regarded as a small range of periodic data, which does not contain cyclical change factors and does not need to be eliminated; The historical time series data after preprocessing to eliminate cyclical variation factors is recorded as the moving average series D.
4. According to claim 3, the method for early warning of high-frequency faults in a data center based on TS-Decomposition is characterized in that: The trend function is fitted by trend inference method, and the long-term trend factors are analyzed. According to the preprocessing results and the obtained long-term trend factors, the irregular change factors are calculated, including: By observing and reasoning, analyzing the trend of the moving average series D obtained by preprocessing, a trend prediction model is selected; the trend prediction model includes a linear trend prediction model and an exponential trend prediction model; Based on the moving average series D obtained by preprocessing, the least squares method is used to fit the parameters of the trend prediction model to obtain the trend prediction function, which is used as the long-term trend factor T t ; Based on the moving average series D and the calculated long-term trend factor T t Calculating Stochastic Volatility Factors 5. According to claim 4, the method for early warning of high-frequency faults in a data center based on TS-Decomposition is characterized in that: The method of establishing a time series prediction model according to the calculation results of the numerical influencing factors, predicting future trends according to the time series prediction model, and obtaining the predicted value of future time series data specifically includes: According to the long-term trend factor T of the historical time series data calculated t , Cyclic change factor C t and irregular variation factor I t Perform trend extrapolation to obtain the long-term trend factor T under the same period / same interval time series t+1 in the future t+1 , Cyclic change factor C t+1 and irregular variation factor I t+1 ; According to the long-term trend factor T in the time series of the same period / same interval in the future t+1 , Cyclic change factor C t+1 and irregular variation factor I t+1 Get the time series prediction model: X t+1 =T t+1 ×C t+1 ×I t+1 The predicted value X of future time series data is obtained by predicting the time series prediction model t+1 .
6. A data center high-frequency fault time domain early warning system based on TS-Decomposition, characterized in that: The system comprises: Data cleaning module: obtains time series monitoring data in the data center scenario and performs data screening and data cleaning to obtain the monitoring values of historical time series data; Data processing module: Analyze the numerical influencing factors of historical time series data, decompose the historical time series data based on the TS-Decomposition algorithm, and calculate the numerical influencing factors based on the monitoring values of the historical time series data; The analyzing the numerical influencing factors of the time series data, performing time series decomposition on the time series data based on the TS-Decomposition algorithm, and calculating the numerical influencing factors according to the monitoring values of the historical time series data specifically includes: The historical time series data after data cleaning includes periodic time series data and interval time series data. For periodic time series data, its numerical influencing factors include: long-term trend factor T t , Cyclic change factor C t and irregular variation factor I t ,Based on the TS-Decomposition algorithm, the multiplication model decomposition of time series data is: X t =T t ×C t ×I t Among them, X t is the total change of the time series, representing the value of the known time series data itself; T t is the long-term trend factor, C t is the cyclical change factor, I t Irregular change factors; For interval time series data, the factors affecting its value include: long-term trend factors, irregular change factors; Preprocess the historical time series data according to the type of time series data to eliminate the cyclic change factor C t Impact on the values of historical time series data; The trend function is fitted using the trend inference method to obtain the long-term trend factor T t , based on the preprocessing results and the obtained long-term trend factors, the irregular change factor I is calculated t , where T t With X t have the same dimension, C t with I t is a ratio; Trend prediction module: establish a time series prediction model based on the calculation results of numerical influencing factors; predict future trends based on the time series prediction model to obtain the predicted value of future time series data; Time domain judgment module: for different time series value types, it performs mixed calculations based on the monitoring values of historical time series data and the predicted values of future time series data to judge the fault time domain of the time series data; For different time series value types, hybrid calculation is performed based on the monitoring values of historical time series data and the predicted values of future time series data to determine the fault time domain of the time series data, specifically including: The monitoring values of historical time series data are recorded as the sequence A{a1,a2,a3,...,an}, and the predicted values of future time series data are recorded as the sequence B{b1,b2,b3,...,bn}. According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on the sequence A and the sequence B to obtain the hybrid sequence C. n , the mixed sequence C n Compare with the lower threshold value and the upper threshold value of the corresponding indicator in different scenarios, and obtain the fault time domain of the time series data according to the comparison result; According to different application scenarios, different hybrid strategies are used to perform hybrid calculations on sequence A and sequence B to obtain a hybrid sequence C. n , specifically including: 1) For periodic time series data whose value fluctuation is less than the first preset threshold, the superposition method is used, and the mixed sequence is recorded as sequence C n ,have C n =A n1 +A n2 +...+A nk +B n Where k is the number of small periods of periodic data, and n is the number of values contained in one period; A nk represents the monitoring value of the historical time series data of the kth small period, B n Represents the predicted value of future time series data; 2) For periodic time series data whose value fluctuation is greater than the second preset threshold, the mean method is used, and the mixed sequence is recorded as sequence C n : A nm Represents the monitoring value of the mth periodic time series data; 3) For time-sensitive periodic time series data, a weighted method is used, and the mixed sequence is recorded as sequence C n : Where k is the number of small periods of periodic data, n is the number of values contained in a period, and λ is the weighting index, which ranges from [0,1]; 4) For periodic time series data whose value fluctuation range is within the preset interval, the peak method is used and the mixed sequence is recorded as sequence C n : C n =max(A n1 ||A n2 ...||A nk ||B n ) The resulting sequence C n It is a sequence of the maximum values of single time nodes in each small period; Fault warning module: Provides time domain warning for high-frequency faults in the data center based on the fault time domain of time series data.
7. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 5.
