Method and system for automatically selecting a model for data stream time series forecasting

Abstract

The present invention provides a method and system for dynamically selecting a prediction model for a time series event. The system includes: statistical logic for generating one or more statistical measures for a set of events in a window; association logic for generating one or more time series association measures for multiple events in the window; model search logic for selecting the best prediction subset of prediction models from multiple prediction models based on a quality measure, wherein the quality measure represents the degree of match between each prediction model and the time series based on the one or more time series association measures; clustering logic for clustering the prediction models in the subset based on the statistical measure; control logic for selecting a prediction model in the subset based on the clustering of the prediction models and the statistical measure; prediction logic for generating prediction outputs in real time from the multiple events in the window using the selected prediction model.

Description

Background Art

[0001] In the current era of big data, data is being generated at extremely high rates and quantities. In many cases, the data is in the form of a time series data stream, making it infeasible to store and query the data offline. As known to those skilled in the art, a data stream is a continuous sequence of events. Each event can be a single data generated by a single source (e.g., a sensor, a machine, or a human), or a combination of data or a pair of data (e.g., an n-tuple) generated by one or more sources. A time series data stream is a data stream in which events are arranged in chronological order. In many applications, it would be valuable to be able to predict the next value or a set of values ​​in a time series data stream in real time, referred to herein as time series data stream prediction or time series data stream forecasting. Examples of such applications include, but are not limited to, predictive maintenance systems, traffic control / monitoring systems, and pollution distribution estimation systems.

[0002] Most data stream processing systems are designed to process a subset of events at a time. Specifically, most data stream processing systems are designed to process a window 102 of events 104 in a time series data stream 106 at any one time, such as Figure 1 As shown. Window 102 is a delimitation of a logical sequence of time or events, including events 104 within the boundary. Events 104 in window 102 generally change over time, as new events arrive and are within the boundary, and old events are outside the boundary and removed from window 102. In this way, a data stream can generate new window instances at different points in time, where the window instance includes events at a specific time instance in the window. That is, a sliding window of variable or fixed size is used to extract a sequence of events from a time series. For example, Figure 1 As shown, sliding window 102 contains or includes the most recent N events in the data stream, where N is an integer greater than or equal to one.

[0003] Performing forecasting on a time series data stream typically includes selecting and / or generating a forecasting model that accurately describes and predicts the time series data stream; and predicting future events from events in a window using the selected forecasting model. In some cases, the forecasting model is selected and parameterized in advance based on known characteristics of the time series data stream. However, the selection and parameterization of the model is typically done through a time-consuming manual process or through an automated process that is resource-greedy and computationally expensive. Furthermore, the advance selection and parameterization of the forecasting model is often not suitable for time series data streams, where its characteristics (e.g., statistical measures) change or evolve over time (e.g., concept drift). Therefore, in order to be able to provide accurate forecasts for a continuously evolving time series data stream, it is desirable to be able to dynamically select and parameterize the forecasting model in real time as the characteristics of the time series data stream change. For example, Figure 1As shown, it is desirable to have a prediction model selector 108 that can dynamically select a prediction model 110 from multiple prediction models 110 for an evolving time series data stream in real time (i.e., with very low latency relative to the time base of the latest event) based on the event 104 currently in the window 102.

[0004] Known systems for processing data streams are used to process data dynamically or in real time. Examples of known systems for processing data streams include, but are not limited to, Apache Flink, Spark Streaming, Storm, Samza, and Dataflow. These systems are used to provide processing capabilities based on the time order of events in the stream. Depending on the specific system, the time can be set to reference event time, processing time, computer time, or arrival time of the event. Many systems allow events to be grouped into windows in some form. However, the window operators of these systems usually work with user-defined functions and are therefore optimized based on function properties. In addition, these systems usually retain all data located in the window range (usually in memory) to be processed by the window function, even if the window function only uses a portion of it. As known to those skilled in the art, a window function is a function applied to a set of events in a window. Exemplary window functions include, but are not limited to, a sorting function or a function that determines the maximum value of events in a window. Although these systems can dynamically process data streams, they do not support dynamic selection of prediction models for data stream time series prediction.

[0005] There are known systems for automatically selecting a forecasting model for a time series and forecasting the time series using the selected forecasting model. These systems are used to determine an appropriate forecasting model for a time series, estimate the parameters of the model, and calculate forecasts based on the model. These systems are generally designed to be able to select forecasting models for a large number of time series that may have unusual time series patterns. The most popular versions of these systems are based on exponential smoothing or AutoRegressive Integrated Moving Average (ARIMA) models. For time series, these systems are generally used to: (i) apply all models that are suitable for the time series and update and optimize the parameters of each model (smoothing parameters and initial state variables); (ii) select the best model among the applied models based on an error metric, such as, but not limited to, Root Mean Square Error (RMSE) or Mean Absolute Percentage Error (MAPE); (iii) generate forecasts for one or more future events; and (iv) obtain prediction intervals for the best model. Examples of these systems are described in "Automatic time series forecasting: The forecast package for R", R. J Hyndman et al., 2008, Journal of Statistical Software, and in "Forecasting with exponential smoothing: the state space approach", R. Hyndman, AB Koehler, J K Ord, and RD Snyder, 2008, Springer Science & Business Media.

[0006] However, these systems have been designed for static time series (e.g., the time series has already been captured, and the model for prediction is selected offline (e.g., not in real time), and these systems are not suitable for real-time prediction of data streams. First, as mentioned above, known time series automatic model selection and prediction systems apply all suitable models to the time series and update and optimize the parameters of each model based on the time series. Updating and optimizing the models requires a time- and resource-intensive convergence process, which may make it difficult to update and optimize many models and predict in real time, especially for high-frequency data streams. Second, time series prediction error metrics, such as RMSE and MAPE, can often be used to select the model with the best prediction accuracy, provided that the error is calculated from a dataset different from the one used for estimation modeling. However, the out-of-sample errors are often too small to draw reliable conclusions.

[0007] Machine learning based systems have been developed for automatically selecting a prediction model for a time series and forecasting the time series using the selected prediction model to address some of the problems of the above statistical systems. Known machine learning based systems for automatically selecting a prediction model for a time series and forecasting the time series using the selected prediction model focus on implementing the selection process of Bayesian integration, support vector regression, nonlinear least squares, ensemble methods, neural networks, and expert systems. An example of a machine learning system is described by R. Fonsca et al. in "Automatic Model Selection in Ensembles for Time Series Forecasting" in the IEEE Transactions on Time Series Forecasting, Vol. 14, No. 8, pp. 3811-3819, August 2016. This paper argues that the diversity and accuracy of the relevant models are the most important factors in selecting a model, and thus describes a system for multi-step forecasting based on a self-organizing map (SOM) neural network and meta-features. The system automatically adjusts the balance between diversity and accuracy in model selection and is able to produce good results for highly nonlinear time series. However, the system requires offline unsupervised training of neural networks and complex model training, making the system unsuitable for implementing real-time model selection and prediction of data streams.

[0008] Another example of a machine learning system is described in “Development and validation of rule-based time series complexity scoring technique to support design of adaptive forecasting DSS” by M. Adya et al., 2016, Decision Support Systems, Volume 83. This paper argues that time series complexity can enable the design of an adaptive forecasting decision support system to actively support the accuracy of forecasting behavior and results. The forecasting decision support system (FDSS) is based on a rule-based complexity scoring technique that generates a complexity score for the time series using 12 rules that depend on 14 features of the series. However, the large number of rules and features makes it difficult to make real-time decisions, making it unsuitable for forecasting time series data streams. In addition, FDSS uses complex features such as discontinuity levels, underlying trend directions, and domain knowledge, which increases the adaptability of the system but also increases its complexity and computation time.

[0009] Therefore, there is a need for systems and methods for automatically selecting a predictive model for a time series data stream in real time that only requires events in a single pass window.

[0010] The embodiments described below are provided by way of example only and are not intended to limit implementations that address any or all of the disadvantages of known systems and methods for data stream time series prediction. Summary of the invention

[0011] The Summary of the Invention introduces some concepts that will be further described in the Detailed Description. The purpose of this Summary of the Invention is to identify key features or essential features of the subject matter protected by the claims, and is not intended to limit the scope of the subject matter protected by the claims.

[0012] Methods and systems for dynamically selecting a prediction model for time series events are described herein. The system includes: statistical logic for generating one or more statistical measures for a set of events in a window; association logic for generating one or more time series association measures for the events in the window; model search logic for selecting a subset of prediction models from a plurality of prediction models based on the degree of match between each prediction model and the time series according to the one or more time series association measures; clustering logic for clustering the prediction models in the subset according to the statistical measures; control logic for selecting one prediction model in the subset according to the clustering of the prediction models and the statistical measures; and prediction logic for generating prediction outputs in real time from the plurality of events in the window using the selected prediction model.

[0013] A first aspect provides a computer-implemented method for dynamically selecting a prediction model for a time series event, wherein the method comprises the following steps in one or more processors: (a) generating one or more statistical measures for a plurality of events in a window, wherein the window comprises a sequence of time series events; (b) generating one or more time series association measures for the plurality of events in the window; (c) selecting a subset of prediction models from a plurality of prediction models based on a quality measure representing a match between each prediction model and the time series according to the one or more time series association measures; (d) clustering the prediction models in the subset according to the statistical measures; (e) selecting a prediction model in the subset according to the clustering of the prediction models and the statistical measures; and (f) generating a prediction output in real time from the plurality of events in the window using the selected prediction model. Steps (a) to (f) may be interleaved in any order, and some of the steps may be performed simultaneously, such as (a) and (b).

[0014] The method may further include repeating (a) to (f) for subsequent windows of the plurality of events in the time series. In the following, all steps (a) to (f) are repeated over a period of time. The calculation is an incremental calculation because the result generated at time t+1 is based on the result at time t. The previous window refers to the event window immediately preceding the window at time t, and the subsequent window refers to the window immediately following the window at time t.

[0015] The one or more statistical measures may be incrementally generated from one or more statistical measures of a previous window of the plurality of events in the time series.

[0016] The one or more time series association metrics may be incrementally generated from one or more time series association metrics of the previous window of the plurality of events in the time series.

[0017] Each statistical metric may describe a mathematical property of the plurality of events in the window.

[0018] The one or more statistical measures include one or more of skewness, kurtosis, standard deviation, and mean of the plurality of events in the window.

[0019] Each time series association metric can describe the relationship between the multiple events in the window according to event parameters. Specifically, a time series event includes parameters such as the time when the event occurs and / or the location where the event occurs.

[0020] The one or more time series association measures may include one or more of a covariance, an autocorrelation function, a partial autocorrelation function, a spatiotemporal covariance function, a spatiotemporal autocorrelation function, and a spatiotemporal partial autocorrelation function.

[0021] The quality measure representing the match between each prediction model and the time series according to the one or more time series association metrics can be based on a criterion that includes a penalty for the number of parameters in the prediction model. In other words, the quality measure represents the degree to which each prediction model matches the time series according to the one or more time series association metrics. The quality measure can be a natural number or a percentage representing the degree of overlap (match) between the prediction model and the time series.

[0022] Determining that the quality measure represents the degree of match between the prediction model and the time series according to the one or more time series association measures may include: combining the statistical properties of the one or more time series association measures and determining the statistical likelihood of the prediction model matching the resulting time series. In other words, each model will provide a prediction of the time series. The quality of the model depends on the quality measure, which in turn may depend on the corresponding statistical likelihood associated with the model. Specifically, the higher the likelihood that the model is "closer" to the real-time series, the higher the value of the quality measure. On the other hand, a low likelihood is an indication of a low quality measure value.

[0023] Determining the quality measure representing the degree of match between each prediction model and the time series according to the one or more time series association metrics may be based on a Bayesian Information Criterion (BIC).

[0024] Clustering the predictive models in the subset according to the statistical measure may include generating a mapping relationship between the one or more statistical measures and a value of each predictive model in the subset.

[0025] The clustering may be fuzzy clustering.

[0026] Selecting one of the prediction models in the subset may be based on a fuzzy interference system.

[0027] The fuzzy interference system may use first order logic statements implementing if-then rules generated from the clusters.

[0028] Steps (a) to (f) may be performed continuously over a sliding window of time series events.

[0029] The time series may be a multi-dimensional time series.

[0030] A second aspect provides a system for dynamically selecting a prediction model for time series events, wherein the system includes: statistical logic for generating one or more statistical metrics for multiple events in a window, wherein the window includes a series of time series events; association logic for generating one or more time series association metrics for the multiple events in the window; model search logic for selecting a subset of prediction models from multiple prediction models based on a quality measurement, wherein the quality measurement represents the degree of match between each prediction model and the time series based on the one or more time series association metrics; clustering logic for clustering the prediction models in the subset based on the statistical metric; control logic for selecting a prediction model in the subset based on the clustering of the prediction models and the statistical metric; and prediction logic for generating a prediction output in real time from the multiple events in the window using the selected prediction model.

[0031] Computer program code for executing the methods described herein may be provided.A non-transitory computer readable storage medium may be provided, having computer readable instructions stored thereon, which when executed at a computer system, causes the computer system to execute the methods described herein.

[0032] The above features may be combined as appropriate, as will be apparent to the skilled person, and may be combined with any aspects of the examples described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Examples will now be described in detail with reference to the accompanying drawings, in which:

[0034] Figure 1 A schematic diagram of a system for dynamically selecting a prediction model for time series prediction of a data stream;

[0035] Figure 2 A schematic diagram for explaining a typical method of calculating window feature operations;

[0036] Figure 3A block diagram of an exemplary implementation of a system for dynamically selecting a prediction model for time series prediction of a data stream;

[0037] FIG. 4A to FIG. 4B Reason Figure 3 Schematic diagram of window processing and data flow orchestration performed by the statistical logic;

[0038] Figure 5 It is a diagram of the autocorrelation function, likelihood function, AIC, and BIC of the example model;

[0039] Figure 6 is a schematic diagram of clustering of three exemplary prediction models according to skewness, mean, standard deviation and kurtosis;

[0040] Figure 7 for Figure 3 A schematic diagram of the high-level operation of the control logic;

[0041] Figure 8 for Figure 3 A schematic diagram of an exemplary implementation of the control logic;

[0042] Fig. 9 for Figure 3 A schematic diagram of the operation of the system in one embodiment;

[0043] Fig.10 A flowchart of an exemplary method for dynamically selecting a prediction model for time series prediction of a data stream;

[0044] Fig.11 For use Figure 3 Schematic diagram of a system performing traffic prediction;

[0045] Fig.12 For use Figure 3 Schematic diagram of a system for performing pollution concentration estimation;

[0046] Fig.13 is a block diagram of an exemplary computing device.

[0047] The accompanying drawings illustrate various examples. It will be appreciated by those skilled in the art that the element boundaries (e.g., boxes, box groups, or other shapes) shown in the accompanying drawings represent an example of boundaries. In some examples, an element may be designed as multiple elements, or multiple elements may be designed as one element. Where appropriate, common reference numerals are used throughout the accompanying drawings to represent similar features. DETAILED DESCRIPTION

[0048] The following description is presented by way of example to enable one skilled in the art to make and use the invention. The invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be apparent to one skilled in the art. The embodiments are described by way of example only.

[0049] Methods and systems for dynamically selecting a prediction model for a time series data stream in real time are described herein, which use a combination of machine learning techniques and statistical analysis. The system includes: statistical logic for generating one or more statistical measures for a set of events in a window; association logic for generating one or more time series association measures for the events in the window; model search logic for selecting a subset of prediction models from a plurality of prediction models based on a quality measure, the quality measure representing the degree of match between each prediction model and the time series according to the one or more time series association measures (i.e., the degree of match between each prediction model and the time series); clustering logic for clustering the prediction models in the subset according to the statistical measure; control logic for selecting one prediction model in the subset based on the clustering of the prediction models and the statistical measure; and prediction logic for generating prediction outputs in real time from the plurality of events in the window using the selected prediction model.

[0050] The systems and methods described herein adaptively adjust model search parameters and models based on metrics (e.g., statistical metrics and time series correlation metrics) generated from incoming events. The systems and methods described herein enable low-latency optimized predictive model search on input data streams, even for high rates of incoming events.

[0051] The systems and methods described herein can model and forecast highly heterogeneous time series and non-stationary spatiotemporal processes in real time. The systems and methods described herein can also exploit spatiotemporal correlations between different, non-stationary, deterministic time series that describe large-scale phenomena. Using data distribution shape measures, time series association measures, and likelihood statistics, the systems and methods described herein can find the parameters and structural configuration of the model that best describes the phenomenon.

[0052] Reference now Figure 3 , shows an exemplary system 300 for dynamically selecting a prediction model for a time series data stream 302 including a series of events 304. The system 300 can be implemented by a computing-based device, such as but not limited to the following combined Fig.13 The computing-based device 1300 is described. For example, there may be computer-readable instructions stored on a computer-readable storage medium that, when executed on the computing-based device, causes the computing-based device to implement Figure 3 system 300.

[0053] System 300 is used to process windows 306 of events 304 of data stream 302 at a time. As described above, a window is a delimitation of a logical sequence of events in time or events, including events within the boundaries. Events 304 located in window 306 typically change over time as new events arrive and are located within the boundaries of the window, and old events are located outside the boundaries and are deleted from the window. In some cases, such as Figure 3 As shown, window 306 can be a sliding window that contains or includes the most recent N events, where N is an integer greater than or equal to 2. System 300 can be used to process events in the window in response to a trigger. In some cases, the trigger may be a change in an event in the window. If the window is a sliding window, the trigger may be the receipt of a new event.

[0054] The system 300 includes: statistical logic 308, used to generate one or more statistical metrics for multiple events in the window 306; association logic 310, used to generate one or more time series association metrics for the multiple events in the window 306; model search logic 312, used to select a subset of prediction models from multiple prediction models based on the degree of match between each prediction model and the time series according to the one or more time series association metrics; clustering logic 308, used to cluster the prediction models in the subset according to the statistical metrics; control logic 310, used to select a prediction model in the subset based on the clustering of the prediction models and the statistical metrics; prediction logic 312, used to generate prediction outputs in real time from the multiple events in the window using the selected prediction model.

[0055] Statistical Logic

[0056] The statistical logic 308, which may also be referred to as a statistical module, is used to generate one or more statistical metrics for the events in the window in real time at each trigger (e.g., each time a new event is received). A statistical metric, which may also be referred to as a statistic or statistical measure, is a mathematical or statistical description of a set of data. Examples of statistical metrics include, but are not limited to, standard deviation, mean / average, skewness, kurtosis, and central moment. Statistical metrics can be used to describe a time series and therefore can be used to select a model that best models or matches the time series. If the statistical properties of the time series change over time (e.g., the time series is non-stationary), periodically measuring or determining one or more statistical metrics helps to dynamically select a prediction model that best matches the current time series in real time.

[0057] Many prior art systems for processing data streams include a window operator that stores all events in a window in memory and processes all events in each trigger (e.g., performs window processing on the events in the window). However, prediction systems for data streams typically use large window sizes (e.g., with millions of events), which (i) requires a large amount of memory to store all events; and (ii) makes it difficult to process all events in the window in (substantially) real time (e.g., at the next trigger or before the trigger). Figure 2 shown.

[0058] Therefore, in some cases, the statistical logic 308 can be used to incrementally generate one or more statistical metrics using an accumulation / backoff method on a sliding window, rather than generating the statistical metrics from scratch each time (e.g., from all events in the window). When the statistical metrics are incrementally generated, the previous statistical metrics (e.g., the statistical metrics of the previous window instance) are used to generate the new statistical metrics (i.e., the statistical metrics of the current window instance).

[0059] Figure 4A An example of how the statistics logic 308 may use the accumulation / backoff method to incrementally generate an average of events in a window is shown at 402. Figure 4A It can be seen that the average value of the n event sets in the window at time t-1 is is the sum of the values ​​of the events xi in the window divided by the total number of events n, as shown in equation (1):

[0060]

[0061] Then, the average value at time t-1 can be obtained according to equation (2): Calculate the average of the set of n events in the window at time t where x(t) is a new event at time t:

[0062]

[0063] This can be implemented using an accumulation / backoff approach where when a new event xa is added to the window, the number n of events in the window at time t+1, the sum s and the average Perform multiple accumulation operations as shown in equations (3), (4), and (5):

[0064] n(t+1)=n(t)+1 (3)

[0065] sn(t+1)=sn(t+1)+xa (4)

[0066]

[0067] When an event xr is removed from the window, multiple back-off operations are performed on the number n, sum s, and average of events in the window at time t+1, as shown in equations (6), (7), and (8):

[0068] n(t+1)=n(t+1)-1 (6)

[0069] sn(t+1)=sn(t+1)-xr (7)

[0070]

[0071] In some cases, the statistical logic 308 can be used to use cache memory and third-party memory in data traffic orchestration or architecture to support low-latency high-throughput calculation of statistical metrics. As known to those skilled in the art, cache memory can also be referred to as central processing unit (CPU) memory, and cache memory is high-speed (usually static random access memory (SRAM)) that the CPU can access faster than conventional random access memory (RAM). In contrast, third-party memory is larger but has slower data storage speeds, such as, but not limited to, hard disks, which are usually farther away from the CPU.

[0072] An example of this type of data traffic orchestration or architecture is Figure 4B 404 is shown in FIG. Figure 4BIn the example shown, the data flow architecture includes a fallback cache 406 and a cumulative cache 408 for storing "hot" events (i.e., events related to the incremental calculation of statistical metrics, such as events that have recently been added to the window and are about to be deleted from the window) and a third-party storage 410, such as a hard drive, for storing "cold" events (i.e., events that are not related to the incremental calculation of the statistical matrix, such as events that have not been recently received in the window or events that will be deleted soon). Specifically, the cumulative cache 408 stores events when they are added to the window. When a new event is added to the cumulative cache 408, the earliest event in the cumulative cache 408 is moved to the third-party storage 410. New events in the cumulative cache 408 can also trigger cumulative events (e.g., can trigger the accumulation operations in equations (3), (4), and (5) to be executed). When a new event is added to the third-party storage 410, the earliest event in the third-party storage 410 is moved to the fallback cache 406. When a new event is added to the back-off cache 406, the oldest event in the back-off cache 406 is removed from the back-off cache 406 that triggers the back-off event (e.g., the back-off operations in equations (6), (7), and (8) may be triggered to be executed). Thus, when an event is a "hot" event (i.e., relevant to the incremental calculation of a statistical metric), it is stored in one of the caches, and when an event is a "cold" event (i.e., not relevant to the incremental calculation of a statistical metric), it is stored in a third-party memory. Such a data traffic orchestration or architecture provides fixed resource allocation for high-frequency input data streams.

[0073] The exemplary data flow arrangement or architecture may also be used to store count or frequency information 412, which indicates the number of current events in the window. For example, if the events are integers, the count or frequency information may indicate the number of "1"s in the current window and the number of "2"s in the current window, etc. The count may be updated when new events are added to the window, and when events are removed from the window. For example, if Figure 4B This is shown as “Case 2”. If the count or frequency information is as shown in Table 1, when a new event “12” is added to the window, the frequency value of “12” is incremented to 2, as shown in Table 2.

[0074] Table 1

[0075] value 1 2 3 12 10 11 frequency 1 3 2 1 1 1

[0076] Table 2

[0077] value 1 2 3 12 10 11 frequency 1 3 2 2 1 1

[0078] In another example, Figure 4BThis is shown as "Case 3". If the count or frequency information is as shown in Table 2 when event "1" is deleted from the window, then since there are no other "1"s in the window, the "1" column is deleted or cleared, as shown in Table 3.

[0079] Table 3

[0080] value 2 3 12 10 11 frequency 3 2 2 1 1

[0081] In another example, Figure 4B This is shown as “Case 1”. If the count or frequency information is as shown in Table 3 when a new event “13” is added to the window, then since there was no “13” originally in the window, a new column is added for “13” and the frequency is set to 1, as shown in Table 4.

[0082] Table 4

[0083] value 2 3 12 10 11 13 frequency 3 2 2 1 1 1

[0084] In another example, Figure 4B As shown in “Case 4”, if the count or frequency information is as shown in Table 4 when event “2” is deleted from the window, the frequency value of “2” decreases from 3 to 2, as shown in Table 5.

[0085] Table 5

[0086] value 2 3 12 10 11 13 frequency 2 2 2 1 1 1

[0087] In some cases, the statistical logic 308 may include a low-latency, high-throughput engine, such as the engine described in the international patent application filed by the applicant with application number PCT / EP2017 / 073956 and entitled "A SYSTEM AND METHOD FOR STREAM PROCESSING" and incorporated herein by reference, for incrementally calculating one or more statistical metrics. Specifically, PCT / EP2017 / 073956 describes a Global Feature Extractor (GFE), which is a stream operator for feature extraction that is capable of incrementally calculating update functions with fixed resources, but supports adaptive optimization to further save resources for hot data for updates (e.g., caching mechanisms) and cold data storage (e.g., disk or third-party storage) for medium- and long-term feature updates. The GFE achieves this goal through optimized function execution that is designed to ensure constant response time for streams with high incoming event rates.

[0088] In some cases, if there are multiple statistical measures, the statistical measures can be generated hierarchically. For example, the mean can be generated using the count, the standard deviation can be generated using the mean, and the kurtosis and skewness can be generated from the standard deviation and the mean by using a closed form incremental formula. This can improve the efficiency of the statistical logic 308.

[0089] Association Logic

[0090] The association logic 310 may also be referred to as an association module, and is used to generate one or more time series association metrics for events in a window at each trigger (e.g., each time a new event is received). A time series association metric is a metric that describes the relationship between a set of data as a function of time and / or space. Exemplary time series association metrics include, but are not limited to, covariance (COV), autocorrelation function (ACF), partial autocorrelation function (PACF), spatial temporal covariance (STCOV), spatiotemporal autocorrelation function (STACF), and spatiotemporal partial autocorrelation function (STPACF). Autocorrelation describes the tendency of near events or observations to be more similar than distant observations, and STACF and STPACF describe the spatiotemporal autocorrelation and partial autocorrelation of time series data between spatial neighbors, respectively.

[0091] Time series correlation measures are often used to design predictors for time series and can therefore be used to select accurate forecasting models for time series. In situations where the time series correlation measures of a data stream change over time, periodically measuring or determining one or more time series correlation measures (e.g., for each window instance) can help dynamically select a forecasting model that accurately predicts the time series.

[0092] In some cases, the association logic 310 is used to generate ACF, STACF and STPACF for each trigger. As known to those skilled in the art, ACF, STACF and STPACF can be used to identify and measure iterative space-time (ST) time series models, such as but not limited to ARIMA models. Specifically, STACF calculates the spatiotemporal autocorrelation of sequence data between s-order neighbors and 0-order neighbors at a time lag t, and the range of spatial lag is from 0 to the length of the weight list (this is a list of weight matrices for each k-order neighbor, the first one is the identifier), and t ranges from 1 to the maximum time lag. STPACF can be calculated by iteratively solving the Yule Walter equation with increasing time lags and spatial lags. It should be noted that if the partial autocorrelation function is not calculated with enough spatial lags, the identification may be biased because the Yule Walker equation is sensitive to a given maximum spatial lag.

[0093] In some cases, the correlation logic 310 is used to generate the time series correlation metric using a theoretical definition of the time series correlation metric. An example theoretical definition of the well-known STCOV that can be used by the correlation logic 310 to calculate STCOV is shown in equation (9):

[0094]

[0095] An exemplary well-known theoretical formula for STACF or STPACF that may be used by the association logic 310 to calculate STACF or STPACF is shown in equation (10):

[0096]

[0097] In some cases, the association logic 310 can be used to use one or more heuristic rules to determine the parameters that the model search logic 312 will use to select and parameterize the relevant prediction model from one or more time series association metrics. For example, in some cases, the association logic 310 can be used to determine the structure of the desired prediction model and the spatial and temporal order based on the shape of the time series association metrics shown in Table 6. For example, if the auto-regression (AR) model is suitable, the ACF and PACF will each have a specific shape or pattern. Specifically, if the AR model is suitable, the ACF will have a weakened infinite damping exponential and / or damped sign wave shape or pattern, and the PACF will have a finite pattern or shape that cuts off after p lags. Similarly, if the moving average (MA) model is suitable, the ACF and PACF will each have a specific shape or pattern. Specifically, if the MA model is suitable, the ACF will be finite and will cut off after q lags, and the PACF will have a weakened infinite damping exponential and / or damped sign wave shape or pattern. Similarly, if an auto-regressive-moving average (ARMA) model is appropriate, the ACF and PACF will each have a specific shape and / or pattern. Specifically, if the ARMA model is appropriate, the ACF and PACF will have the shape of a damped infinitely damped exponential and / or damped sign wave. As shown in Table 6, the maximum AR and MA orders p and q can be determined from the cutoff values ​​of the PACF and ACF, respectively.

[0098] Table 6

[0099]

[0100] In some cases, the correlation logic 310 may include a low-latency, high-throughput engine, such as the engine described in international patent application number PCT / EP2017 / 073956 filed by the applicant and entitled “A SYSTEM AND METHOD FOR STREAM PROCESSING”, which is incorporated herein by reference, for incrementally computing one or more time series correlation metrics.

[0101] Model search logic

[0102] The model search logic 312 may also be referred to as a model search module, and is used to select a subset of prediction models from a plurality of prediction models based on one or more time series association metrics calculated by the association logic 310. In some cases, the model search logic 312 may be used to receive search parameters from the association logic 310, parameterize appropriate prediction models based on the search parameters, and then select a subset of parameterized prediction models based on the degree of match between each prediction model and the time series. For example, as described above, the association logic 310 may be used to determine the structure and spatial and temporal order of the desired prediction model based on the time series association metric. For example, the desired structure (e.g., AR, MA, or ARMA) may be selected from the shapes of the PACF and ACF, and the maximum MA and AR orders may be selected from the PACF and ACF cutoffs, respectively, which may be provided to the model search logic 312 as search parameters. Given these search parameters (e.g., structure and order), the model search logic 312 estimates the parameters of a prediction model having the desired structure. The estimation process may infer the residuals of the underlying response sequence and fit the prediction model to the response data by maximum likelihood to generate a parameterized prediction model. The quality metric represents the quality of the fit, ie, how well the prediction model matches the time series. The model search logic 312 then selects a subset of parameterized prediction models based on how well each prediction model matches the time series according to one or more time series association metrics.

[0103] In some cases, the model search logic 312 is used to determine the degree of match of each parameterized prediction model to the time series based on a likelihood measure based on the one or more time series association measures. As described above, many known systems for automatic model selection of (non-stream) time series use standard forecast or prediction metrics or measurements, such as but not limited to Root Mean Squared Error (RMSE) or Mean Absolute Percentage Error (MAPE), but these metrics cannot select between error types when the point forecasts (i.e., predictions) from two models are the same. However, metrics based on likelihood rather than one-step forecasts are able to select between error types.

[0104] In some cases, the likelihood measure X of the model can be calculated according to equation (11), where L is the maximized log-likelihood objective function of the model:

[0105]

[0106] in,

[0107]

[0108] In some cases, the model search logic 312 is used to determine how well each of the multiple prediction models matches the time series based on a likelihood metric that includes a penalty for the number of parameters in the prediction model. When fitting a model to a time series, the likelihood can be increased by adding parameters, but doing so may result in overfitting. As known to those skilled in the art, overfitting is the production of a model that is too close or corresponds exactly to a set of data, making it difficult to reliably accurately predict future events or data points. Therefore, the use of a likelihood metric that penalizes the number of parameters significantly reduces the risk of selecting an overfit model. Penalized likelihood metrics include, but are not limited to, Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC).

[0109] AIC compares models from the perspective of information entropy, measured by the Kullback-Leibler divergence. The AIC for a given model can be calculated from the log-likelihood L according to equation (13), where k is the number of parameters in the model:

[0110]

[0111] BIC is also known as the Schwarz Information Criterion and is closely related to AIC. BIC compares models from a decision theory perspective, measured by expected loss. The BIC of a given model can be calculated according to equation (14), where k is the number of parameters in the model and N is the number of data points:

[0112]

[0113] BIC is an estimate of the function of the posterior probability of the model being true in a specific Bayesian setting, so the lower the BIC, the more likely the model is to be the true model. Due to the larger penalty term in BIC, BIC may be better at ensuring that overfitting models are not selected. BIC can also provide useful information about the model, regardless of the type of predictive model.

[0114] In some cases, the AIC or BIC can be generated incrementally. Specifically, in some cases, the log-likelihood can be calculated incrementally by using a Jacobian free iterative estimation process that does not perform the typical (non-incremental) costly covariance matrix calculations. In other cases, the log-likelihood can be calculated incrementally using a neural network-based incremental log-likelihood estimator that is a wavelet-based radial basis function network. This estimator is similar to the classic radial basis function (RBF) network, but the radial basis functions are multi-scale wavelets and scaling functions.

[0115] Figure 5 An exemplary ACF, likelihood L, and AIC and BIC for an exemplary time series are shown. In this example, the ACF is used to generate the likelihood, which is used to generate the AIC and / or BIC.

[0116] In some cases, the model search logic 312 can be used to select W prediction models that best match the time series based on the time series association metric, where W is an integer greater than or equal to 2. In some cases, the model search logic 312 can be used to select three prediction models with the lowest BIC. However, for those skilled in the art, this is just an example, and other numbers of prediction models can be selected.

[0117] Clustering logic

[0118] The clustering logic 314 may also be referred to as a clustering module, and is used to cluster the subset of prediction models selected by the model search logic 312 according to the statistical measure generated by the statistical logic 308. As known to those skilled in the art, clustering or cluster analysis is the process of grouping a group of objects in a manner that makes objects in the same group more similar to each other than objects in other groups. The clustering logic 314 is used to cluster or group the selected subset of prediction models according to the statistical measure, so that all models describing a certain range of values ​​of the statistical measure will be in the same cluster. Therefore, each cluster groups the prediction models according to the degree to which the prediction models represent the statistical measure. By performing clustering, the clustering logic 308 effectively learns the mapping between the statistical measure and the selected subset of prediction models. The clustering logic 308 can output one or more clustering configuration parameters or metrics that describe how the selected subset of prediction models are clustered.

[0119] In some cases, fuzzy clustering is used to cluster subsets of predictive models. In non-fuzzy clustering, which can also be called hard clustering, the data is divided into different clusters, where each data point can only belong to one cluster. In contrast, in fuzzy clustering (which can be called soft clustering), a data point may belong to multiple clusters. Since in model selection, the technical solution may not be unique (for example, the model parameters and structure of multiple models may be the same), fuzzy clustering can be implemented so that a data point can belong to multiple clusters.

[0120] In some cases, clustering logic 308 can be used to cluster subsets of the prediction model using a fuzzy-C mean (FCM) clustering algorithm. As known to those skilled in the art, the FCM algorithm is an iterative algorithm that works by assigning memberships to data points, which represent the degree to which the data point belongs to each cluster. In FCM, membership is based on the distance between the data point and the cluster center or centroid. In FCM, the center or centroid of a cluster is the mean of all points, weighted by its membership to the cluster. The algorithm first generates an initial set of memberships (e.g., a matrix) and calculates the initial fuzzy cluster center from it. In each iteration, the cluster center and membership are updated. The objective function is to find the best position for the cluster. When the maximum number of iterations is reached or the objective function improvement between two consecutive iterations is less than the specified minimum improvement, the iterative process stops. In these cases, the cluster configuration parameters or metrics output by clustering logic 308 may include the membership of each data point (i.e., each statistical metric) and the cluster center.

[0121] For example, clustering logic 308 can be used to output U and C. U is a fuzzy segmentation matrix including n_clusters rows and data_len columns, where n_clusters is the number of models and data_len is the number of statistical measures. Element U(i,j) represents the membership of the jth statistical measure in the i-th cluster (i.e., in the i-th model). For a given statistical measure, the sum of the membership values ​​of all clusters is 1. C is a cluster center matrix including n_clusters rows and one column for each dimension of the clustered data. Each element contains the coordinates of the cluster center of the corresponding statistical measure. It is obvious to those skilled in the art that this is just an example and other clustering algorithms can be used to cluster a selected subset of the prediction model.

[0122] Figure 6 An example of how clustering logic 314 clusters an exemplary set of three predictive models when clustering logic 314 implements FCM clustering and receives skewness, kurtosis, standard deviation (std), and mean as statistical measures is shown in FIG. Figure 6 It is shown that as the statistical measure changes, the cluster assignments also change. Specifically, from Figure 6 It can be seen that at time t, the samples (skewness, kurtosis, std, and mean) are in cluster 1 and cluster 3 (i.e., model 1 or model 3), and at time t+1, the samples (skewness, kurtosis, standard deviation, and mean) are in cluster 1 and cluster 2 (i.e., model 1 or model 2).

[0123] Control Logic

[0124] The control logic 316 may also be referred to as a control module or automatic model selector for selecting one of the prediction models in the subset based on the clustering and statistical measures of the prediction models (in real time). In other words, the control logic 316 is used to select one of the selected prediction model subsets for time series prediction of the data stream based on the output of the clustering logic 314 (e.g., information describing how the selected model subset is clustered based on the statistical measure) and the output of the statistical logic 308 (e.g., one or more statistical measures). Figure 7 A high level representation of control logic 316 is shown.

[0125] In some cases, the control logic 316 can be used to convert the output of the clustering logic 314 (e.g., the membership / fuzzy segmentation matrix) into a set of rules (e.g., if-then rules) that describe how to select the best prediction model from the statistical metrics and how to select one of the prediction models in the subset from the statistical metrics based on the rule set. The rule set can use the membership functions extracted in the clustering and can be parameterized based on the cluster centers and the segmentation matrix.

[0126] In some cases, the control logic 316 is used to use a fuzzy inference system / controller (FIS / FIC) to select the prediction model that best fits the time series described by one or more statistical metrics. As known to those skilled in the art, fuzzy inference is the process of using fuzzy logic to develop a mapping from a given input to an output. The mapping then provides a basis from which decisions can be made, or patterns can be identified. Figure 8An exemplary FIS / FIC architecture for implementing control logic 316 is shown in FIG. In this example, control logic 316 includes fuzzification logic 802, inference engine 804, and defuzzification logic 806. Fuzzification logic 802 receives one or more statistical measures (e.g., skew, kurtosis, standard deviation (std), and mean) generated by statistical logic 308 and fuzzifies them. As known to those skilled in the art, fuzzification is the process of changing an actual scalar value (i.e., a clear value) into a fuzzy value. This can be achieved by different fuzzifiers (also referred to as membership functions). An example of a membership function is a Gaussian curve. Inference engine 804 receives the fuzzy statistical measures generated by fuzzification logic 802 and uses a set of rules 808 to generate a fuzzy output that indicates which of the selected prediction model subsets best matches the statistical measure.

[0127] Rule set 808 is generated from the output of clustering logic 814 (eg, the distance between the membership value indicating the value and the cluster center). In some cases, rule set 808 may be in the form of a first-order-logic (FOL) statement that formulates an IF-THEN rule. Figure 8 An exemplary rule set 810 is shown in FIG. Defuzzification logic 806 is used to convert the fuzzy output generated by the inference engine 804 into a crisp output representing the predictive model that best matches the statistical metric.

[0128] Control logic 316 is used for continuous operation. Since the calculation of descriptive measures (e.g., statistical measures and time series association measures) is updated in real time, adjusting the separation boundaries between models accordingly also occurs continuously, facilitating adaptively and dynamically selecting the model that best matches the time series.

[0129] General Overview

[0130] Reference now Fig. 9 , provides how the logic 308, 310, 312, 316, 318 of the above system 300 work together when the system is running to provide automatic real-time selection of a prediction model for time series prediction of the data stream 902. Specifically, as Fig. 9As shown, the statistical logic 308 and the association logic 310 generate (in 904) one or more statistical measures (e.g., slope, kurtosis, std, and mean) and one or more time series association measures (e.g., STACF and STPACF) that are incremented on the data stream; the model search logic 312 parameterizes the prediction model based on the time series association measure, and selects a subset of the parameterized prediction models that best matches the time series based on an information criterion such as BIC or AIC. In 906, the clustering logic 314 calculates the cluster membership of the subset of statistical measures and parameterized prediction models based on, for example, FCM clustering, and outputs the clustering measure, for example, a fuzzy partition matrix and cluster centers as the data stream advances. In 908, the control logic 316 uses a fuzzy inference system or controller to select one of the parameterized prediction models based on the clustering measure and the statistical measure. The selected parameterized prediction model is then used to generate a prediction of the data stream based on the events in the window.

[0131] method

[0132] Reference now Fig.10 , showing a method for dynamically selecting a prediction model for a Figure 3 An exemplary method 1000 for performing time series forecasting on a data stream of a series of events implemented by the system 300 of FIG. The time series data stream may be non-stationary, deterministic, and / or multi-dimensional. The method 1000 may be implemented by a computing-based device, such as, but not limited to, the following in combination with Fig.13 The computing-based device 1300 described herein may include, for example, a non-transitory computer-readable storage medium having computer-readable instructions stored thereon, which, when executed on the computing-based device, causes the computing-based device to implement Fig.10 system 1000.

[0133] Method 1000 begins at steps 1002 and 1004, generating one or more statistical measures for the set of events in the window, and generating one or more time series association measures for the set of events in the window, respectively. Statistical measures describe mathematical statistics or properties of events in the window. Exemplary statistical measures include, but are not limited to, skewness, kurtosis, standard deviation, and mean of events in the window. Time series association measures describe the relationship between events in the window as a function of time and / or space. Examples of time series association measures include, but are not limited to, covariance, autocorrelation function, partial autocorrelation function, space-time covariance function, space-time autocorrelation function, and space-time partial autocorrelation function. In some cases, one or more statistical measures and / or one or more time series association measures are incrementally generated (i.e., based on previous values ​​of the measures).

[0134] In step 1006, a subset of prediction models is selected from a plurality of prediction models based on the degree of match between each prediction model according to the one or more time series association metrics and the time series. In some cases, the degree of match between each prediction model according to the one or more time series association metrics and the time series is based on a criterion including a penalty based on the number of parameters in the prediction model. In some cases, the degree of match between the prediction model according to the one or more time series association metrics and the time series includes: combining the statistical properties of the one or more time series association metrics and determining the statistical likelihood of the prediction model matching the resulting time series. Examples of likelihood criteria including penalties based on the number of parameters include, but are not limited to, Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC).

[0135] In step 1008, the predictive models in the subset are clustered according to one or more statistical measures. In some cases, clustering the predictive models in the subset according to the statistical measures includes generating a mapping relationship between the one or more statistical measures and the value of each predictive model in the subset. In some cases, the clustering is fuzzy (or soft) clustering.

[0136] In step 1010, a prediction model in the subset is selected based on the clustering of the subset of prediction models and the statistical measure. In some cases, the selection of a prediction model in the subset can be based on a fuzzy interference system. An example of a fuzzy interference system is described above. In these cases, the fuzzy interference system can use first-order logic statements that implement if-then rules generated from the clustering.

[0137] In step 1012, a prediction output is generated in real time from the plurality of events in the window using the selected prediction model.

[0138] In some cases, steps 1002 to 1012 may be iteratively repeated for subsequent instances of the window so as to adaptively select a prediction model based on the current characteristics of the data stream. For example, in some cases, steps 1002 to 1012 may be performed continuously on a sliding window of time series events.

[0139] Test Data

[0140] The above-described system 300 for dynamically selecting a predictive model for performing time series forecasting on a data stream has been tested in several real-world scenarios where the process generating the data stream is unknown (i.e., there is no information about how the data was generated) and there are effects of highly nonlinear time series evolution (indicative of a non-stationary and deterministic time series).

[0141] Reference now Fig.11 , shows the output (i.e., predicted value) of the above system 300 when used for traffic data prediction. Traffic congestion poses a serious challenge to urban infrastructure and also affects the socio-economic life of urban residents due to time wasted in traffic. Many known prediction systems fail to accurately predict traffic flow because they fail to fully utilize the topological structure and intrinsic time flow of vehicles and their correlation.

[0142] The aim of the test is to generate a traffic forecast through the intersection or intersection A a full day in advance given the traffic profiles of intersections A, C, D, E, F and H recorded in the past 20 days, e.g. Fig.11 For testing, the system 300 is used to select from a set of spatio-temporal auto regressive moving average (STARMA) models, including STAR, STMA, and STARMA. The system 300 is capable of automatically selecting the best prediction model structure and its parameters by exploiting the spatio-temporal correlations between multiple connected intersections in a city.

[0143] from Fig.11 It can be seen that the system 300 is able to find and parameterize the best fitting model according to MAPE and use it to predict the 51st data after showing a 20-day traffic profile starting from the 30th day in a data set of 56 data, with 288 readings per day (i.e., one reading every 5 minutes).

[0144] Reference now Fig.12 , shows the output (ie, predicted value) of the above system 300 when used for pollution data prediction. According to the World Health Organization (WHO), air pollution accounts for a large part of the total mortality rate.

[0145] The purpose of the test is to predict the pollution sensor of one station 30 days in advance based on the pollution sensor readings recorded by 8 stations in the past 590 days. For the test, the system 300 is used to select from a set of Spatio-Temporal Auto Regressive Moving Average (STARMA) series models, including: STAR, STMA and STARMA. The system 300 is able to automatically select the best prediction model structure and its parameters by exploiting the spatiotemporal correlation of multiple stations measuring gases (e.g., SO2, NO2, O3), meteorology (e.g., temperature, pressure) and pollution (e.g., PME10, PM25).

[0146] from Fig.12 It can be seen that the system 300 is able to find and parameterize the best fitting model according to MAPE and use the model to predict contamination at a particular site.

[0147] Exemplary Computing-Based Devices

[0148] Fig.13 Various components are shown of an exemplary computing-based device 1300, which may be implemented as any form of computing and / or electronic device and in which embodiments of the methods and systems described herein may be implemented.

[0149] The computing-based device 1300 includes one or more processors 1302, which may be microprocessors, controllers, or any other suitable type of processors, for processing computer executable instructions to control the operation of the device so as to dynamically select a prediction model for time series prediction of a data stream. In some examples, for example, where a system-on-chip architecture is used, the processor 1302 may include one or more fixed function blocks (also referred to as accelerators) that implement a portion of a method for verifying a hardware design of a data conversion pipeline in hardware (rather than software or firmware). An operating system 1304 or any other suitable platform software may be provided at the computing-based device to enable application software to be executed on the device.

[0150] Computer executable instructions may be provided using any computer-readable medium accessible to the computing-based device 1300. Computer-readable media may include, for example, computer storage media, such as memory 1306 and communication media. Computer storage media (i.e., non-transient machine-readable media), such as memory 1306, include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (digital versatile disk, DVD) or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store access by a computing device. In contrast, communication media may include computer-readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transmission mechanism. As defined herein, computer storage media does not include communication media. Although computer storage media (ie, non-transitory machine-readable media such as memory 1306) are shown within computing-based device 1300, it will be appreciated that the memory may be remotely distributed or located and accessed via a network or other communications link (eg, using communications interface 1308).

[0151] The computing-based device 1300 also includes an input / output controller 1310, which is used to output display information to a display device 1312, which can be separate from or integrated into the computing-based device 1300. The display information can provide a graphical user interface. The input / output controller 1310 is also used to receive and process input from one or more user input devices 1314 (such as a mouse or keyboard) and other devices. This user input can be used to start the method or system. In one embodiment, if the display device 1312 is a touch-sensitive display device, the display device 1312 can also serve as the user input device 1314. The input / output controller 1310 can also output data to devices other than the display device, such as a locally connected printing device ( Fig.13 not shown).

[0152] Applicants hereby disclose separately each individual feature described herein and any combination of two or more such features. With the common knowledge of those skilled in the art, such features or combinations can be implemented as a whole according to this specification, without considering whether such features or combinations of features can solve any problem disclosed herein. In view of the above description, it will be obvious to those skilled in the art that various modifications can be made within the scope of the present invention.

Claims

1. A computer-implemented method for dynamically selecting a prediction model for time series events, characterized in that: The method is implemented in one or more processors using a data flow architecture including an accumulation cache, a third-party memory, and a fallback cache; wherein time series events input to the data flow architecture pass through the accumulation cache, the third-party memory, and the fallback cache in a first-in-first-out manner, and the method comprises the following steps: (a) generating one or more statistical measures for a plurality of events in a window, wherein the window comprises a series of the time series events; the one or more statistical measures are generated based on a global feature extractor computing an update function to incrementally compute one or more statistical measures of a previous window of the plurality of events in the time series; the one or more statistical measures comprise one or more of skewness, kurtosis, standard deviation and mean of the plurality of events in the window; the global feature extractor is a stream operator for feature extraction; (b) generating one or more time series association metrics for the plurality of events in the window; the one or more time series association metrics are incrementally generated from one or more time series association metrics of the previous window of the plurality of events in the time series; the one or more time series association metrics include one or more of a covariance, an autocorrelation function, a partial autocorrelation function, a spatiotemporal covariance function, a spatiotemporal autocorrelation function, and a spatiotemporal partial autocorrelation function; (c) selecting a subset of prediction models from the plurality of prediction models based on a quality measure representing a degree of match between each prediction model and the time series based on the one or more time series association metrics; (d) clustering the prediction models in the subset according to the statistical measure; (e) selecting a prediction model in the subset based on the clustering of the prediction models and the statistical measure; (f) generating a prediction output in real time from the plurality of events in the window using the selected prediction model; The generating one or more statistical metrics for a plurality of events in a window, wherein the window includes a series of the time series events, comprises: storing new events in the accumulation cache, moving the oldest events in the accumulation cache to the third-party storage, and triggering an accumulation event; The earliest event in the third-party storage is moved to the fallback cache, the earliest event in the fallback cache is deleted, and a fallback event is triggered.

2. The method according to claim 1, characterized in that Also included is repeating (a) through (f) for subsequent windows of the plurality of events in the time series.

3. The method according to claim 1 or 2, characterized in that: The quality measure representing the degree of match between each prediction model and the time series according to the one or more time series association metrics is based on a criterion including a penalty for the number of parameters in the prediction model.

4. The method according to claim 1 or 2, characterized in that: Determining that the quality measure represents the degree of match between the prediction model and the time series based on the one or more time series association metrics includes: combining statistical properties of the one or more time series association metrics and determining the statistical likelihood of the prediction model matching the resulting time series.

5. The method according to claim 1 or 2, characterized in that: Determining the quality measure representing the degree of match between each prediction model and the time series according to the one or more time series association metrics is based on a Bayesian Information Criterion (BIC).

6. The method according to claim 1 or 2, characterized in that: Clustering the predictive models in the subset according to the statistical measure includes generating a mapping relationship between the one or more statistical measures and a value of each predictive model in the subset.

7. The method according to claim 1 or 2, characterized in that: The clustering is fuzzy clustering.

8. The method according to claim 1 or 2, characterized in that: Selecting one of the prediction models in the subset is based on a fuzzy interference system.

9. The method according to claim 8, characterized in that The fuzzy interference system uses first order logic statements implementing if-then rules generated from the clusters.

10. The method according to claim 1 or 2, characterized in that: (a) to (f) are performed continuously on the sliding window of the time series events.

11. The method according to claim 1 or 2, characterized in that: The time series is a multidimensional time series.

12. A system for dynamically selecting a prediction model for time series events, characterized in that: The system is implemented using a data flow architecture including an accumulation cache, a third-party memory, and a fallback cache; wherein time series events input to the data flow architecture pass through the accumulation cache, the third-party memory, and the fallback cache in sequence in a first-in-first-out manner, and the system includes: Statistical logic for generating one or more statistical measures for a plurality of events in a window, wherein the window comprises a sequence of time series events; the one or more statistical measures are generated based on a global feature extractor computing an update function based on one or more statistical measure increments of a previous window of the plurality of events in the time series; the one or more statistical measures comprising one or more of skewness, kurtosis, standard deviation, and mean for the plurality of events in the window; the global feature extractor being a stream operator for feature extraction; association logic for generating one or more time series association metrics for the plurality of events in the window; the one or more time series association metrics being incrementally generated from one or more time series association metrics of the previous window for the plurality of events in the time series; the one or more time series association metrics comprising one or more of a covariance, an autocorrelation function, a partial autocorrelation function, a spatiotemporal covariance function, a spatiotemporal autocorrelation function, and a spatiotemporal partial autocorrelation function; model search logic for selecting a subset of prediction models from the plurality of prediction models based on a quality measure representing how well each prediction model matches the time series based on the one or more time series association metrics; clustering logic for clustering the prediction models in the subset according to the statistical measure; control logic for selecting a prediction model in the subset based on the clustering of the prediction models and the statistical measure; prediction logic for generating a prediction output in real time from the plurality of events in the window using the selected prediction model; The generating one or more statistical metrics for a plurality of events in a window, wherein the window includes a series of the time series events, comprises: storing new events in the accumulation cache, moving the oldest events in the accumulation cache to the third-party storage, and triggering an accumulation event; The earliest event in the third-party storage is moved to the fallback cache, the earliest event in the fallback cache is deleted, and a fallback event is triggered.

Images