Automated Deep Learning Architecture Selection for Time Series Forecasting with User Interaction

Abstract

A system and method for automatically generating deep neural network architectures for time series forecasting includes a processor that performs the following steps: receiving a forecasting context associated with a current use case; selecting a predictive model network configured for the time series forecasting task of the current use case based on the associated forecasting context; replicating the selected predictive model network to create multiple candidate predictive model networks; inputting time series data to each of the multiple candidate predictive model networks; training each of the multiple candidate predictive model networks in parallel with the input time series data; modifying each of the multiple candidate predictive model networks by applying a different set of one or more model parameters while training in parallel; and determining a modified predictive model network that best fits the time series forecasting task of the current use case.

Description

[Technical Field]

[0001] The present invention relates to automated machine learning (AutoML) and cloud computing, and more particularly to methods and systems for automatically analyzing time series data to build and deploy time series forecasting models. [Background technology]

[0002] Many types of scientific and business data include time series data (e.g., IoT, retail, transportation, energy / renewables). Time series data share similar characteristics: multi-level seasonality, highly correlated features, noise, and large volume.

[0003] Currently, the use of deep neural networks (DNNs) has shown high accuracy for many complex time series forecasting applications, especially when multi-level seasonality is not easily separable.

[0004] Furthermore, there exist cloud-based service entities that offer online provisioning of computing resources as a service, in particular cloud-based AutoML (automated machine learning) services where artificial intelligence or machine learning models are generated and built by and for end-user customers. Summary of the Invention

[0005] The following summary is for illustrative purposes only and is not intended to limit the scope of the claims.

[0006] According to one aspect, a system and method are provided for a cloud-based service provider to automatically build and deploy cloud-based time series forecasting models.

[0007] According to a further aspect, a system and method for building an automated DNN architecture selection method and system for time series forecasting with user interaction is provided. In one embodiment, the automated DNN architecture selection method and system for time series forecasting is an extension to automated machine learning systems for time series and tools that allow for the creation of time series prediction pipelines in a fully automated manner.

[0008] According to one aspect, a computer-implemented method for automatically synthesizing a deep neural network architecture optimized for time-series forecasting is provided, the method including: receiving, at a hardware processor, a forecasting context associated with a current use case; using the hardware processor to select an existing predictive model network architecture configured for a similar use-case time-series forecasting task based on the associated forecasting context; replicating the selected existing predictive model network architecture to create a plurality of candidate predictive model network architectures; using the hardware processor to input time series data to each of the plurality of candidate predictive model network architectures; training, in parallel, each of the plurality of candidate predictive model network architectures with the input time series data; modifying, in parallel, each of the plurality of candidate predictive model network architectures by applying a different set of one or more model parameters while training in parallel; and determining, in parallel, a modified predictive model network architecture from the plurality of candidate predictive model network architectures that is best suited to solving the time-series forecasting task of the current use case.

[0009] According to one aspect, a computer-implemented system for automatically synthesizing deep neural network architectures optimized for time-series forecasting is provided, comprising: a memory storage device for storing a computer-readable program; and at least one processor adapted to execute the computer-readable program, the computer-readable program configuring the at least one processor to: receive a forecasting context associated with a current use case, select an existing predictive model network architecture configured for a similar use-case time-series forecasting task based on the associated forecasting context, replicate the selected existing predictive model network architecture to create a plurality of candidate predictive model network architectures, input time series data to each of the plurality of candidate predictive model network architectures, train each of the plurality of candidate predictive model network architectures in parallel with the input time series data, modify each of the plurality of candidate predictive model network architectures by applying a respective different set of one or more model parameters while training in parallel, and determine, from the plurality of candidate predictive model network architectures, a modified predictive model network architecture that is best suited to solving the time-series forecasting task of the current use case.

[0010] In a further aspect, a computer program product for performing operations is provided, the computer program product including a storage medium readable by a processing circuit and storing instructions for execution by the processing circuit to perform a method, the method being the same as described above. [Brief explanation of the drawings]

[0011] The above aspects and other features are described below in conjunction with the accompanying drawings, in which: [Figure 1]FIG. 1 is a schematic diagram of an exemplary data processing framework applicable for implementing embodiments for automatically building a time series forecasting model, according to embodiments of the present invention. [Figure 2] FIG. 1 is a block diagram conceptually illustrating the components of an AutoML-TS plug-in module. [Figure 3A] FIG. 1 illustrates an example of a pluggable business use case for a short-term electricity demand forecasting model generated to forecast future time series data. [Figure 3B] FIG. 1 illustrates a portion of an example multi-layer perceptron (MLP) feed-forward architecture structure for use as an initial or pluggable business use case architecture, according to one embodiment of the present invention. [Figure 4] FIG. 1 illustrates a method performed by an architecture search module to grow a new DNN model from a selected architecture, according to one embodiment of the present invention. [Figure 5] FIG. 5 illustrates the growth of a DNN time series forecasting model using the N-Cevolve search algorithm of FIG. 4. [Figure 6] FIG. 1 illustrates a table providing a mapping between types of DNN model architectures (e.g., CNN, RNN, MLP) and subsets of mutations applicable to a particular DNN architecture type. [Figure 7A] FIG. 1 illustrates a user interface displayed on a user device that enables data entry for the automated construction of a time series forecasting model. [Figure 7B] FIG. 1 illustrates a user interface that presents a recommended model architecture pipeline for time series forecasting in response to a business use context input via a user. [Figure 7C] FIG. 10 illustrates a user interface that presents recommended pipelines depending on a business use context entered via a drop-down menu selection. [Figure 8]FIG. 1 is a schematic diagram of an example computer or processing system capable of implementing a method for automatically synthesizing and training a deep neural network architecture optimized for time series forecasting, according to one implementation of the present invention. [Figure 9] FIG. 1 illustrates a cloud computing environment according to one embodiment of the present invention. [Figure 10] FIG. 2 illustrates abstraction model layers according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] According to one embodiment, the present disclosure provides a system and method for system providers to perform time series (TS) forecasting, in particular, an automated, general-purpose data processing framework that achieves higher accuracy for complex time series forecasting problems.

[0013] That is, the (multivariate) time series X={ x (1),..., x (n)} (where: x Given a time series (t)={x(t),...,x(t)}), the system and method synthesizes and trains an optimized deep neural network (DNN) architecture configured to perform time series prediction.

[0014] 1 is a general block diagram illustrating an automated data processing framework 100 for achieving higher accuracy for complex time series prediction problems. The automated general framework is configured to achieve higher accuracy for complex time series prediction problems. These complex time series prediction problems include prediction problems where: 1) a DL architecture exists for predicting time series; 2) there is a large variance in real time and the knowledge series (e.g., dependency structure, noise level, data volume, etc.) is limited; or 3) additional parameters to adjust (e.g., window length (for fixed-size images), window overlap, etc.) exist.

[0015] In one embodiment, the automated data processing framework 100 runs on one or more computing systems 102, receives inputs including time series data, and builds machine learning models to solve time series forecasting problems. In one embodiment, an end user, e.g., a data scientist 101, can upload time series data 104 to an object data store 115. For example, via a user interface 110, the end user uploads data such as time series data (e.g., tables) to a cloud object store. Alternatively, the end user 101 can specify that the automation framework 100 receive the time series data from a web-based storage location specified by a uniform resource locator (URL). Other data inputs to the system include, but are not limited to, the forecast target and predictor, timestamp / date, task (forecasting (including forecasting horizon), anomaly detection, clustering, etc.), task-related parameters (lookback window, window overlap, etc.), required DNN algorithm (e.g., CNN, RNN, etc.), domain of time series data (e.g., financial services, supply chain, transportation, etc.), and existing DNN architectures used in past related use cases.

[0016] In one embodiment, the end user 101 can further interact with a notebook 120, such as Watson Studio (a registered trademark of International Business Machines Corporation) or any similar web-based application for interactive computing 106. For example, using the notebook software tool 120, the end user 101 can use the Python library of application programming interfaces (APIs) 125 to assemble, test, and execute all the building blocks for processing data and store the data in object storage 115 for use in building and deploying time series forecasting models. In one embodiment, the notebook 120 can include a Jupyter notebook application that enables interactive document creation and sharing and executes methods for data cleaning / transformation / analysis, statistical modeling, machine learning, etc. Other similar data analysis and visualization notebook tools 120 that execute the Python, Julia, and R programming languages ​​can also be used to receive and process data, context, and other inputs to the processing framework 100.

[0017] 1 , the processing framework 100 includes an AutoML-TS construction process or plug-in module 150 that interfaces with the notebook implementation 120 program and Python library to construct a time-series forecasting model 140 according to an embodiment of the present invention. In one embodiment, the automated machine learning for time-series data (AutoML-TS) construction process or plug-in module 150 consumes data input in comma-separated value (CSV) format 110 from the notebook 120 and / or an object storage cloud. In one embodiment, a data model 109 is invoked to convert raw time-series data stored in tables and other input formats in object storage into a comma-separated value (CSV) format for use by the AutoML-TS construction process (plug-in) 150. The AutoML-TS plug-in module 150 includes a model search module 130. The model search module 130 automatically executes a method for first searching relevant model architectures for time series data for a particular domain, business problem, or context, extracting the best model architecture, and invoking a further machine learning module 140 for building a concrete, consumable time series forecasting model from the extracted best model architecture in accordance with the principles of the present invention.

[0018] 1, once a specific, consumable time series forecasting model is constructed, it can be deployed via the cloud data processing framework 100 as a web-based application or downloaded to a specific user device 105. In one embodiment, a Flask application 135 provides a user interface (e.g., a Web Server Gateway Interface) that allows end users, via their computing devices 115, to forward requests to use the deployed AutoML-TS and other framework modules written in Python over a public or private network 99.

[0019] A block diagram illustrating the operational components of the AutoML-TS plug-in module 150 is shown in FIG.

[0020] As shown in FIG. 2, the AutoML-TS plug-in module 150 of the automated data processing framework 100 builds models to solve time series forecasting problems.

[0021] In one embodiment, the AutoML-TS plug-in module 150 of the automation framework includes an initial DNN architecture storage library 175 containing model building architectures that can be used as a starting point for building new DNN models according to time-series data and other user inputs. To achieve this, the framework 100 further includes a time-series DNN architecture search module 170 that executes program code implementing logic for searching for matches with existing use-case model architectures. That is, an initial search is performed to determine whether similar time-series data has been processed by the AutoML-TS plug-in module 150 and whether a time-series model prediction architecture has already been created for that data to solve the time-series prediction problem. If the time-series DNN architecture search module 170 finds that a use case exists for a similar problem or time-series data context (e.g., short-term energy forecasting or credit card fraud prediction), the search module 170 selects the corresponding architecture for that use case as the initial architecture for the initial DNN architecture search library 175.

[0022] In one embodiment, as shown in FIG. 2 , the automated AutoML-TS plug-in module 150 of the data processing framework 100 further includes a module 165 for storing pluggable pipelines of business use cases. The pluggable business use case pipeline storage module 165 includes all previously built DNN prediction pipeline architectures, i.e., historical use cases of DNNs built to solve time series prediction problems for historical time series prediction contexts, including, but not limited to, short-term electricity demand forecasting, credit card fraud detection, traffic volume forecasting, IoT building anomaly detection, financial credit risk prediction, and transportation parking availability prediction. Essentially, the pluggable business use case storage module 165 is an architecture and a database of past business use cases associated with previous contexts. When a user provides the context of a particular time series dataset, the system can use the pluggable pipeline storage module 165 to discover all previously built DNN prediction pipeline architectures that provide optimal solutions.

[0023] In one embodiment, the AutoML-TS plug-in 150 further includes a transfer / meta learner module 160 for architecture selection. For example, for newly input time series data corresponding to the current input context, the pluggable pipeline storage module 165 first determines whether an identical or similar previous / past use case context exists, i.e., whether a DNN pipeline architecture has already been built for that particular context. If no use case exists and a DNN architecture pipeline has not been built for that type of time series data, the meta / transfer learning module 160 for architecture selection executes program code that implements logic for calculating a list of characteristic features of the time series data (e.g., variability, skewness, kurtosis, trend, seasonality, and Hurst constant (i.e., describing a measure of long-term memory for a time series)). Based on the list of characteristic features characterizing the input time series data, the meta / transfer learning module 160 can discover a recommended initial DNN architecture. In one embodiment, meta-learner module 160 is an ML learning model that can take in input features such as skewness, kurtosis, etc., and output a starting neural network architecture for the purpose of identifying similarities between user input data and data in the system.

[0024] In one embodiment, the meta-learning module 160 can be enabled / disabled by the user. A user can enable meta-learning if they believe that the dataset they input is typical of one that the system has likely encountered in the past.

[0025] FIG. 3A illustrates an example pluggable business use case and meta-learning embodiment. In particular, the pluggable business use case module 165 provides an inventory of top-picked architectures based on past use cases. One pluggable business use case 200 shown in FIG. 3A analyzes multivariate time series data 210 (X t ) to receive previously generated and future time series data 215 (X t+1 The short-term power demand forecasting model 205 is used to forecast the short-term power demand. The short-term power demand forecasting model 205 is a DNN model architecture including an RNN and an LSTM with 100 hidden layers. This architecture 200 is output as a recommendation output from the module 165 when a user has the same input use case.

[0026] As a further example of a selected architecture pipeline, Figure 3B is a schematic diagram of a portion of an exemplary multi-layer perceptron (MLP) feedforward architecture structure 250. In the exemplary MLP feedforward neural network model architecture structure 250 shown in Figure 3B, a linear configuration of multiple (successive) layers, e.g., input layer, hidden layer, and output layer, is shown. Each layer receives input information, performs some computation, and ultimately outputs transformed information. The output of one layer flows into the next layer as its input. In the exemplary DNN MLP architecture structure 250, the top layer 253 indicates an identifier for a particular successive MLP model. Subsequent layers include a first flattening layer 256 for flattening the input data to the CNN model, one or more conventional deeply connected neural network or dense layers 260, one or more leaky ReLU activation function layers 265, which are piecewise linear functions modified to allow small negative values ​​when the input is less than zero, and one or more dropout regularization layers 270 for reducing overfitting. Each of these layers includes a representation 280 of the shape of the input to that layer and a representation 290 of the output of that layer. These layers include layers that function to process input univariate or multivariate time series datasets. Other layer types that can be included in a CNN or RNN model architecture include, but are not limited to, cropping layers, concatenation layers, convolutional layers, and pooling / unpooling layers. To fully specify each layer of the selected architecture, additional details such as (but not limited to) the shape of the input data, the number of neurons / units in the layer, and any initializers, regularizers, or constraints can be specified.

[0027] Such a structure / dataset architecture 250 is associated with past use in a particular context (e.g., credit card fraud detection) and stored as a pluggable pipeline for future use in time series forecasting when new input data for a similar context is presented. Thus, when a new dataset for this context is presented as user input, the system starts from the initial architecture 250 shown in Figure 3B and builds a new model for the current new input dataset.

[0028] According to embodiments of the present invention, the initial model architecture can be adjusted taking into account the characteristic features of a new input dataset (e.g., univariate or multivariate) and the selected or specified type of DNN architecture. For example, for univariate time series input data including a single series of observations with a time order, a DNN modeling architecture is formed that learns from a series of past observations to predict the next value in the sequence. For multivariate time series input data, an architecture for DNN modeling of multiple input time series data having two or more parallel input time series, each having an observation at the same time step, and an output time series that depends on the input time series, or an architecture for modeling multiple parallel time series inputs, each of which needs to have a value predicted, is searched for.

[0029] According to an embodiment of the present invention, each stored DNN modeling architecture can be searched, i.e., mutated, to create other complex neural networks to optimize a particular prediction problem or otherwise tailor the solution to a particular runtime or prediction accuracy. That is, based on a determination from the meta-learning module 160 that a similar context (e.g., credit card fraud risk prediction) existed in the past, the time-series DNN architecture search module 170 initiates a new search (i.e., applies mutations to the selected architecture) for a previously constructed architecture relevant to that context (i.e., credit card fraud risk prediction). The meta-learning module 160 does not recommend an initial architecture if the use case does not exist in the database. However, given the context and characteristics of the data, it is possible to derive assumptions for an initial architecture to be used as a starting point.

[0030] 6 illustrates a table 600 that provides a mapping between types of DNN model architectures (e.g., CNN, RNN, MLP) and possible mutations 650 that can be applied to a particular DNN architecture type. For example, as shown in FIG. 6, an initial MLP DNN architecture 602 may initially include a series of layers 612, e.g., Input, Flatten, Dense (linear), etc. Meanwhile, an initial CNN architecture 605 may include a series of layers 615, e.g., Input, Convolution, MaxPooling, Flatten, Dense, etc.

[0031] Returning to FIG. 2, the TS DNN architecture search module 170 executes a method for synthesizing a new TS DNN architecture from a selected or initial DNN architecture. For example, the search module 170 can execute a program such as NCEvolve, an evolutionary algorithm that implements a method for synthesizing a top-performing network while minimizing training time and resource requirements. As described in M. Wistuba, "Deep Learning Architecture Search by Neuro-Cell-Based Evolution with Function-Preserving Mutations" (ECML-PKDD 2018), the entire contents and disclosure of which are incorporated herein by reference, NCEvolve synthesizes CNN architectures for images / text by mutating neuro-cells using function-preserving operations. Function-preserving operations are a collection of network operations for transferring knowledge from one network to another. The TS DNN architecture search module 170 utilizes NCEvolve and extends it for time-series data by adding an MLP structure. For example, starting from an initial DNN architecture, NCEvolve applies a series of predefined mutations (e.g., inserting CNN layers, changing kernel size and branching) until the learning curve no longer improves.

[0032] For example, as shown in FIG. 6, a selection from predefined operations can be performed on the initial DNN architecture based on characteristic features of the input data or user-selected parameters (e.g., execution time, accuracy, etc.). For example, in the case of an MLP architecture, such operations or mutations 625 applicable to the architecture include, but are not limited to, changing the window length, changing the overlap size, inserting a dense layer, changing the learning rate, changing the number of units fed into a layer, etc. For example, in the case of an initial MLP model architecture, function-preserving mutations 652 that can be applied can include expanding the shape of the input data or inserting a dense layer. In the case of a CNN model architecture, function-preserving mutations 654 that can be applied can include changing the identity, changing the kernel size, inserting a convolutional layer, changing the learning rate, changing the number of channels, branching and insertion, branching and insertion of blocks, inserting a residual block, changing the number of units fed into a layer, branching convolution, changing the window length, changing the overlap size, inserting a dense layer, inserting a dense layer dropout, or inserting multiple filters. For example, for highly varianced data with unknown temporal dependencies, the "window length" parameter of the model can be expanded or varied multiple times. In the case of an RNN architecture 606, mutations 656 that can be applied to the architecture can include changing the window length, changing the overlap size, inserting a dense layer, changing the learning rate, or changing the number of units fed into a layer.

[0033] Returning to Figure 2, in one embodiment, the DNN search module 170 runs the N-Cevolve (Neural Cell Evolution) algorithm to grow a new DNN model from the selected architecture. The algorithm starts with an initial architecture and sequentially applies different mutations (e.g., layer insertion, branching, etc.). Automated DNN search is applied to time-series data, but because the data is a sequence and has time dependencies, the algorithm must group them in a way that preserves the order and preserves the time dependencies.

[0034] More specifically, the NCEvolve approach starts with a very simple network template containing a sequence of neural cells. These neural cells are architectural patterns, and the optimal pattern is to be automatically discovered by the AutoML-TS algorithm. The algorithm assumes that the cell initially contains only a single convolutional layer and then continues to modify it through function-preserving mutations. These mutations alter the structure of the architecture without changing the network's predictions. This can be thought of as a special initialization that makes the network less computationally intensive to train.

[0035] (Evolution of neuro-cells) Cell-based neuroevolution works as follows: Given a very simple neural network architecture containing multiple neural cells, the cells themselves share their structure, and the task is to find a structure that improves the overall neural network architecture for a given dataset and machine learning task. Initially, the cells can be the same as convolutional layers and are modified during an evolutionary optimization process. The evolutionary algorithm selects individuals from a population through tournament selection, where a subset k of individuals is randomly selected from the population. From this set, the individual with the highest fitness is selected for mutation. In one embodiment, fitness is determined by the accuracy achieved by the individual on a hold-out dataset. Mutations are randomly selected and applied to all neural cells, maintaining cell identity. The network is trained on the training set for several epochs and then added to the population. Finally, the process starts over. After meeting several stopping criteria, the individual with the highest fitness is returned.

[0036] (mutation) All mutations used are based on function-preserving algorithms, i.e., mutations do not change the fitness of the individuals but increase their complexity. The advantage compared to creating the same network structure with randomly initialized weights is that a partially pretrained network is used as the starting point. This allows the network to be trained in fewer epochs. All mutations are applied only to the structure within the neuronal cells. The following mutations are considered in the neuroevolutionary algorithm:

[0037] Convolution insertion: A convolution is added at a random position. Its kernel size is, for example, 3x3, and the number of filters is the same as the input dimensions. Alternatively, a separable convolution may be used, which is determined randomly.

[0038] Convolution branching and insertion: A convolution is randomly selected and branched. In one of the branches, a new convolution is added according to the "convolution insertion" mutation.

[0039] Skip insertion: A convolution is randomly selected, and its output is added to the output of a newly added convolution (see "Convolution Insertion"), which then becomes the input to the next layer.

[0040] Changing the number of filters: A convolution is randomly selected and augmented by a factor sampled uniformly at random from, say, [1.2,2]. This variation may be applied to convolutions outside the neuron.

[0041] Change the number of units: Similar to the above, but change the number of units in the fully connected layer. This mutation only applies to the outside of the neuron.

[0042] Change kernel size: Randomly select a convolution and increase its kernel size by 2 along each axis.

[0043] Convolutional branching: A convolution is randomly selected and branched. The motivation for choosing this set of mutations is to allow the neuroevolutionary algorithm to discover architectures similar to those proposed by human experts. By adding convolutions and combining skipping and adding convolutions, a residual network can be discovered. Finally, a combination of branching, kernel size changes, and adding (separable) convolutions allows for the discovery of architectures similar to Inception, Xception, and FractalNet. The optimization starts with only one individual. The population is enriched by starting with an initialization step that creates 15 mutated versions of the first individual. Individuals are then selected based on the tournament selection process described above.

[0044] FIG. 4 illustrates a method 400 performed by the architecture search module 170 to grow a new DNN model from a selected architecture using the NCEvolve algorithm.

[0045] Initially, in step 405, an empty "pool" is defined. Operations are dragged into the pool and applied to the selected original or initial DNN model architecture. Next, in step 410, an "individuum" or "blocking structure" model architecture is trained and mutations are applied to "grow" a new model. In one embodiment, multiple individuals or model architectures are trained in parallel, with different mutations applied to blocks in each individual to improve the speed at which the best-fitting TS prediction model is obtained. Growing a new model from the original DNN model architecture in step 410 can include applying a series of mutations to blocks, such as, but not limited to, inserting flatter data layers, inserting dense layers, changing (e.g., widening) the window length, changing the overlap length, changing model hyperparameters, branching and inserting, changing the kernel size, inserting convolutional layers, and increasing the learning rate.

[0046] Next, the method continues to step 415, where it is determined whether a predetermined training time has elapsed for training the individual. By way of example, a 24-hour training time may be set for training the model, although this training time is user-configurable. If the training time has not elapsed, then in step 425, a "fittest" individual is selected from the random subset of individuals, and in step 430, the selected fittest individual is subjected to additional random mutations for fine-tuning. The fittest individual is the model with the lowest computational error when run on a test dataset and the prediction error determined within a predetermined accuracy. The method then returns to step 415, where it is again determined whether the predetermined training time has elapsed. If not, process steps 425-430 are repeated. If it is determined that the predetermined training time has elapsed, then in step 420, the method outputs the trained fittest individual.

[0047] FIG. 5 illustrates the growth of a DNN time-series forecasting model 500 using the N-Cevolve search algorithm 400 of FIG. 4. In particular, the N-Cevolve search algorithm operates on an individual 501 having a block structure 502A, 502B, ..., 502N, which is evaluated using a training dataset. In an embodiment, the DNN search algorithm is fully automated and time-series specific. The DNN search algorithm is tailored for time-series forecasting using a wide range of DNN architectures, including, for example, CNNs, RNNs, and MLPs, as well as concatenations of "block" structures, including, for example, CNNs and MLPs. Each of the block structures 502A, 502B, ..., 502N mimics ensemble methods in ML. This eliminates the previous (incorrect) assumption that only recurrent network models should be applied to time-series data. The method begins with multiple "blocks" 502A, 502B, ..., 502N. Each block of an individual 501 contains a neural network (NN) itself. Instead of fitting a single NN to an input time series dataset 510, which may include multivariate time series data 512, multiple parallel NNs are fitted to the input data 510 simultaneously in parallel blocks. Each block structure 502A, 502B, ..., 502N resembles an ensemble model, including, for example, an MLP, a CNN, an RNN, or a combination thereof, making the architecture more robust and enabling parallel search. For example, as shown, block 502A of the individual 501 contains only an MLP, while block 502B contains connections from both an MLP and a CNN. As shown in step 410 of FIG. 4, different mutations can be applied in parallel to each block 502A, 502B, ..., 502N of the individual. One or more individuals 501 are then processed in parallel.For example, as shown in Figure 5, a first block structure 502A can be modified with a first window length (WL_1) parameter 505A, block 502B can be modified with a second WL_2 parameter 505B, and the Nth block 502N can be modified with a WL_N parameter 505N. For blocks implementing CNNs, convolutional layers may be inserted, branch and insert operations may be performed, and kernel sizes may be changed. Returning to Figure 6, in particular, mutation subsets 652, 654, 656 from the superset of mutations 625 may be applied sequentially to each parallel trained block 502A, ..., 502N.

[0048] In one embodiment, the type of mutation selected to apply to a particular block is data-driven. A particular mutation is applied based on the characteristics of the data and the type of DNN network structure of the block structure. For example, calculated characteristics of the input time series data are first obtained and / or provided. The characteristics include, for example, variability, skewness, kurtosis, trend, seasonality, and the Hurst constant value (i.e., a measure of time dependency). These data characteristics are then used to determine the components of the mutation sequence to be applied. For example, a window length mutation parameter is determined based on the Hurst constant value.

[0049] Returning to Figure 5, each block 502A, 502B, ..., 502N can generate an output that can be combined in a combination layer 520. The combination layer 520 generates a single prediction for the input data set. The output can be the average of the outputs of each individual block 502A, 502B, ..., 502N. For example, the final output layer of a block structure, such as the MLP feedforward structure shown in Figure 3A or 3B, provides an output prediction that is combined in the combination block 520.

[0050] The trained individual 501 with the best fitness or lowest prediction error is output as the new optimized DNN model for the new input dataset / context.

[0051] Returning to Figure 2, the output of the architecture search module 170 is a new architecture with best-fitting blocks for solving, for example, a credit card fraud risk detection model. That is, the AutoML-TS plug-in module 150 of the data processing framework 100 outputs a newly created, trained, and optimized DNN architecture 180 for time series forecasting, regardless of whether an initial architecture recommendation was provided. In addition to outputting the trained and optimized DNN architecture, any predictions / anomaly lists / clusters (task-dependent) are generated along with uncertainty scores and optional explanations (e.g., for extreme events) for presentation or display via the device user interface output.

[0052] 2, the AutoML-TS plug-in module 150 of the data processing framework 100 further includes a user interface 110 that enables user interaction with the architecture. Through the user interface, an end user can interact with the architecture to initiate a search and further define the desired run time. The user can pause the search at any time to obtain intermediate results.

[0053] 7A illustrates a user interface presented on a user device that enables data entry for automated construction of a time-series forecasting model. For example, prior to using method 400 of FIG. 4, a user can add a data source via interface screen 700, such as by dragging and dropping a CVS file into the interface's receptacle, or browse for an upload file up to a predetermined size, in portion 703. Alternatively, a user can specify or load data and models for a particular project in portion 706.

[0054] Figure 7 Bfurther illustrates in interface screen 700 a Yes / No selection 710 that provides the ability to create a time series forecast to forecast future activity over a specified date / time range. Additionally, a drop-down menu selection 720 is provided to allow user input of business use context. That is, selecting a forecast column identifies a model architecture (pluggable pipeline) that can, for example, predict values ​​(e.g., profit and loss) of one column (of a data table) based on the values ​​of the remaining columns.

[0055] Figure 7 C FIG. 7 illustrates a user interface 701 that presents recommended pipelines in response to a business use context entered via a drop-down menu selection 720. C In the example, the user interface may select a typical machine learning pipeline 730 (e.g., autoregressor, XGBoost, Holt-Winters) or one or more existing deep neural network architectures (e.g., profit_loss AutoML TS). 740 Here is a recommended pipeline including:

[0056] Figure 7 B further illustrates a menu selection 725 that allows a user to select a date / time column (e.g., transactionDate) via a user interface presented on the user device. In response to selecting a date / time column via menu selection 725, a user can enter a lookback time window via selection 750 of interface screen 700 that specifies how far back to trace transaction time series data. Additionally, a drop-down menu selection 775 is provided that allows a user to select a forecast time window (e.g., one day ahead).

[0057] Returning to FIG. 2 , via an additional user interface 155, the user can input the desired run time for solving the time series forecasting problem and the desired accuracy of the time series forecast results (forecast). Alternatively, the user can specify a run time ratio preference, which is the ratio of additional accuracy gain to run time. For example, the accuracy gain is 1% per hour, meaning that adding one hour of run time leads to a 1% increase in accuracy, conditional on the current accuracy. The system uses the run time accuracy to determine whether to run further or terminate. Additionally, the user can input system constraints. Some user-input constraints can include requirements (e.g., features) or, if the current architecture is different, a preferred DNN model (e.g., MLP, CNN, RNN). User-input constraints can further include a maximum training time, a model execution time, or a selected target accuracy score. The architecture search stops when the user goal is reached. Alternatively, the user can enter a command to force the process to stop, for example, to obtain intermediate results.

[0058] Through a further interface 185, the user can receive the results of the time series prediction problem (e.g., intermediate or final results) along with an explanation. For example, after a DNN search is complete, the user can request a final output with an explanation. The final output is the optimal neural network architecture and insight into the features (e.g., what are the top two dominant features in this architecture). For example, if a user wants to predict energy consumption, the system 100 can output a CNN with MLP blocks, and the explanation could be, "Convolutional layers account for X% of this architecture, and temperature and cloud coverage account for Y% of the data variability," where "temperature" and "cloud coverage" are features from the input data. The explanation is a by-product of training and optimizing the DNN module. In one embodiment, this is implemented by an extension of the Local Interpretable Model-agnostic Explanations (LIME) library, available at https: / / github.com / marcotcr / lime. Such explanations may further include a list of causal explanations for the outcome and possible explanations if the output is an outlier based on the uncertainty score.

[0059] As an example, we present experimental results between Deep AutoML TS and the current state of the art in terms of predictive ability.

[0060] Example 1: DNN AutoML TS represents the output architecture compared to state-of-the-art methods (autoregressive moving average (ARMA) and Holt-Winters forecasting models) for two datasets in the experiment. In particular, this example shows forecasting univariate time series data. 1) The root mean square error (RMSE) results based on a model with known order (e.g., ARMA(1,1)+T(5)) using the ARMA(1,1) approach were 1.34, while the RMSE using AutoML-TS with CNN was 1.11. Based on Trend+Seasonality+ARMA(1,1)+N(0,1), the Holt-Winters error was 1.58, while the RMSE using AutoML-TS with CNN was 1.29. As the results show, AutoML-TS has a smaller forecast error.

[0061] FIG. 8 illustrates an exemplary computing system consistent with the present invention. It should be noted that the illustrated computer system is merely one example of a suitable processing system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the present invention. For example, the illustrated system is operational with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with the system illustrated in FIG. 8 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable appliances, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of these systems or devices.

[0062] In some embodiments, the computer system may be implemented as program modules stored in memory 16 and described in the general context of computer system-executable instructions executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks and / or implement particular input data and / or data types in accordance with the present invention (see, e.g., FIGS. 2 and 4).

[0063] Components of a computer system may include, but are not limited to, one or more processors or processing units 12, a system memory 16, and a bus 14 operatively connecting various system components, including the system memory 16, to the processor 12. In some embodiments, the processor 12 may execute one or more modules 11 loaded from the memory 16, where the program modules embody software (program instructions) that cause the processor to perform one or more method embodiments of the present invention. In some embodiments, the modules 11 may be programmed into integrated circuits of the processor 12, or may be loaded from the memory 16, a storage device 18, or a network 24, or a combination thereof.

[0064] Bus 14 may represent any one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an Accelerated Graphics Port (AGP), and a processor or local bus using any of a variety of bus architectures. By way of non-limiting example, such architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0065] The computer system may include a variety of computer system readable media, which may be any available media that can be accessed by the computer system and may include both volatile and nonvolatile media, removable and non-removable media.

[0066] Memory 16 (sometimes referred to as system memory) may include computer-readable media such as volatile memory (such as RAM, cache memory, or other forms or combinations thereof). The computer system may also include other removable / non-removable, volatile / non-volatile computer-readable media. By way of example only, storage system 18 may be provided for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Also, although not shown, storage system 18 may include a magnetic disk drive for reading from and writing to removable, non-volatile magnetic disks (e.g., a "floppy disk") and an optical disk drive for reading from and writing to removable, non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media). In these examples, each may be connected to bus 14 by one or more data media interfaces.

[0067] The computer system may also communicate with one or more external devices 26, such as a keyboard, pointing device, display 28, one or more devices that allow a user to interact with the computer system, or any device that allows the computer system to communicate with one or more other computing devices (e.g., a network card, modem, etc.), or a combination thereof. Such communication may occur via an input / output (I / O) interface 20.

[0068] Additionally, the computer system may communicate with one or more networks 24, such as a local area network (LAN), a general-purpose wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter 22. As shown, the network adapter 22 may communicate with other components of the computer system via a bus 14. Although not shown, other hardware and / or software components may be used with the computer system, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0069] The present invention may be a system, method, or computer program product, or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium having stored thereon computer-readable program instructions for causing a processor to carry out aspects of the present invention.

[0070] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROMs, DVDs, memory sticks, floppy disks, punch cards, or mechanically encoded devices having instructions recorded on ridge-in-groove structures, and the like, and suitable combinations thereof. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted over a wire.

[0071] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing device / processing device. Alternatively, they can be downloaded to an external computer or external storage device via a network (e.g., the Internet, a LAN, a WAN, or a wireless network, or a combination thereof). The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing device / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions to a computer-readable storage medium in the respective computing device / processing device for storage.

[0072] The computer-readable program instructions for carrying out the operations of the present invention can be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as the "C" programming language and similar programming languages. The computer-readable program instructions can execute entirely on the user's computer as a stand-alone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a LAN or WAN, or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to customize the electronic circuitry for carrying out aspects of the present invention.

[0073] Aspects of the present invention are described herein with reference to flowchart and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. Each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer-readable program instructions.

[0074] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, whereby the instructions, executed by the processor of such computer or other programmable data processing apparatus, create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner. The computer-readable storage medium having instructions stored thereon thereby constitutes an article of manufacture including instructions for performing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0075] Computer-readable program instructions may also be loaded into a computer, other programmable apparatus, or other device and a series of operational steps executed on the computer, other programmable apparatus, or other device to create a computer-implemented process, whereby the instructions executing on the computer, other programmable apparatus, or other device perform the functions / operations identified in one or more blocks in the flowcharts and / or block diagrams.

[0076] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for performing specific logical functions. In some other implementations, the functions shown in the blocks may be executed in an order different from that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may be executed in reverse order, depending on the functionality involved. Note that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs specific functions or operations or executes a combination of dedicated hardware and computer instructions.

[0077] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. Furthermore, as used herein, the terms "comprises" and / or "comprising" specify the presence of stated features, integers, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof. Corresponding structure, materials, acts, and equivalents of all elements in the following claims are intended to include any structure, material, or act for performing the function as specifically claimed in combination with other claimed elements. The description of the present invention has been presented for purposes of illustration and description and is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The present embodiments have been chosen and described in order to best explain the principles and practical applications of the invention and to enable others skilled in the art to understand the invention in various embodiments with various modifications suitable for the particular uses contemplated.

[0078] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention may be practiced in conjunction with any other type of computing environment now known or later developed.

[0079] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model includes at least five characteristics, at least three service models, and at least four deployment models.

[0080] The characteristics are as follows: On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time or network storage, automatically as needed, without the need for human interaction with the service provider. Broad network access: Computing power is available over the network and can be accessed through standard mechanisms, facilitating use by heterogeneous thin or thick client platforms (e.g., cell phones, laptops, PDAs). Resource Pooling: Computing resources from a provider are pooled and offered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated based on demand. Consumers generally have no control or knowledge of the exact location of the resources they are provided with, resulting in a sense of location independence. However, consumers may be able to determine location at a higher level of abstraction (e.g., country, state, data center). Rapid Elasticity: Computing capacity can be provisioned quickly and elastically, sometimes automatically, to instantly scale out and quickly release to instantly scale in. To the consumer, the computing power available for provisioning often appears unlimited, and can be purchased at any time and in any quantity. Metered Services: Cloud systems leverage measurement capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user accounts) to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of utilized services.

[0081] The service model is as follows: Software as a Service (SaaS): The functionality offered to the consumer is the availability of a provider's applications running on a cloud infrastructure that can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., webmail). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functionality, except for limited user-specific application configuration settings. Platform as a Service (PaaS): The capability offered to consumers is to deploy applications they create or acquire using programming languages ​​and tools supported by the provider onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of their hosting environment. Infrastructure as a Service (IaaS): The functionality offered to consumers is the provisioning of processors, storage, networking, and other basic computing resources on which they can deploy and run any software, including operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating system, storage, and deployed applications, and in some cases partial control over some network components (e.g., host firewalls).

[0082] The deployment model is as follows: Private Cloud: This cloud infrastructure is dedicated to a specific organization and can be managed by that organization or a third party, and can exist on-premise or off-premise. Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common concerns (e.g., mission, security requirements, policies, and compliance). This cloud infrastructure can be managed by those organizations or a third party and can exist on-premises or off-premises. Public cloud: This cloud infrastructure is available to the general public or large industry organizations and is owned by an organization that sells cloud services. Hybrid cloud: This cloud infrastructure combines two or more cloud models (private, community, or public), each of which retains its inherent nuances but is bound by standards or specific technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0083] A cloud computing environment is a service-oriented environment that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0084] FIG. 9 illustrates an exemplary cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10, with which local computing devices used by cloud consumers (e.g., PDAs or cell phones 54A, desktop computers 54B, laptop computers 54C, or automobile computer systems 54N, or combinations thereof) can communicate. The nodes 10 can communicate with each other. The nodes 10 can be physically or virtually grouped (not shown) in one or more networks, such as the private, community, public, or hybrid clouds described above, or combinations thereof. This enables the cloud computing environment 50 to provide infrastructure, platform, or software as a service, or combinations thereof, for which cloud consumers are not required to maintain resources on their local computing devices. It should be understood that the types of computing devices 54A-N illustrated in FIG. 9 are merely exemplary, and that the computing nodes 10 and the cloud computing environment 50 can communicate with any type of electronic device via any type of network or network-addressable connection (e.g., using a web browser), or both.

[0085] A set of functional abstraction layers provided by the cloud computing environment 50 (FIG. 9) is now shown in FIG. 10. It should be understood in advance that the components, layers, and functions shown in FIG. 10 are merely exemplary, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0086] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage device 65, and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0087] The virtualization layer 70 provides an abstraction layer from which virtual entities such as virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75 can be provided.

[0088] By way of example, the management layer 80 may provide the following functionality: Resource provisioning 81 enables dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 enables cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. By way of example, these resources may include application software licenses. Security enables identification and verification of cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 enables allocation and management of cloud computing resources so that requested service levels are met. Service level agreement (SLA) planning and fulfillment 85 enables advance arrangement and procurement of anticipated future cloud computing resources required in accordance with SLAs.

[0089] The workload layer 90 provides examples of functionality available in a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and automatically synthesizing and training optimized deep neural network architectures, including convolutional neural networks, recurrent neural networks, and multilayer perceptrons, for time series forecasting 96.

Claims

1. 1. A computer-implemented method for automatically synthesizing optimized deep neural network architectures for time series forecasting, comprising: receiving, at a hardware processor, a prediction context associated with a current use case; selecting an existing predictive model network architecture configured for a similar use case time series forecasting task using the hardware processor based on the associated forecasting context; and replicating the selected existing predictive model network architecture to generate a plurality of candidate predictive model network architectures; inputting time series data into each of the plurality of candidate predictive model network architectures using the hardware processor; training each of the plurality of candidate predictive model network architectures in parallel using the input time series data using the hardware processor; modifying each of the plurality of candidate predictive model network architectures by applying a different set of one or more model parameters, including a different lookback time window length parameter and a different window overlap parameter, while training in parallel using the hardware processor; using the hardware processor to determine, from among the plurality of candidate predictive model network architectures, a modified predictive model network architecture that is best suited to solving the time series forecasting task for the current use case; and 11. A computer-implemented method comprising:

2. 2. The computer-implemented method of claim 1, wherein the predictive model network architecture comprises one or more deep neural network models for time series forecasting selected from a convolutional neural network, a recurrent neural network, and a multi-layer perceptron neural network, or a combination thereof.

3. 10. The computer-implemented method of claim 1, wherein selecting a predictive model network architecture for the current use case comprises searching a library of existing predictive model network architectures previously generated for past use cases related to the current prediction context.

4. obtaining characteristic features of the input time series data using the hardware processor; using the hardware processor to identify similarities between the characteristic features of the input time series data and features of data used as input to an existing predictive model network architecture; using the hardware processor to select the existing predictive model network architecture configured for a time series forecasting task of a current use case based on the identified similarities; and The computer-implemented method of claim 1 , further comprising:

5. 3. The computer-implemented method of claim 2, further comprising: accepting, at the hardware processor, via a user interface, user constraints for performing the time series forecasting task of the current use case, wherein the one or more user constraints include at least one or more of a specific predictive model network architecture, a model training time limit, a prediction accuracy, or an execution time ratio.

6. outputting the best-fit predictive model network architecture via the user interface; and presenting, via the user interface, a description of one or more dominant features in the best-fit predictive model network architecture; and The computer-implemented method of claim 5 further comprising:

7. The plurality of candidate predictive model network architectures to be trained comprises individuals, and the method further comprises: generating a plurality of individuals using a hardware processor; For each individual, modifying each of the plurality of candidate predictive model network architectures by applying a different set of one or more model parameters while training in parallel using the hardware processor; The computer-implemented method of claim 2 further comprising:

8. selecting a random set of individuals using said hardware processor; using the hardware processor to determine the fittest individual from the random set; applying random mutations to one or more candidate predictive model network architectures of the determined fittest individuals using the hardware processor; and The computer-implemented method of claim 7 further comprising:

9. 1. A computer-implemented system for automatically synthesizing optimized deep neural network architectures for time series forecasting, comprising: a memory storage device for storing a computer readable program; at least one processor adapted to execute the computer readable program, the computer readable program causing the at least one processor to: receiving a prediction context related to a current use case; selecting an existing predictive model network architecture configured for a similar use case time series forecasting task based on the relevant forecasting context; and replicating the selected existing predictive model network architecture to generate a plurality of candidate predictive model network architectures; inputting time series data for each of the plurality of candidate predictive model network architectures; training each of the plurality of candidate predictive model network architectures in parallel using the input time series data; modifying, while training in parallel, each of the plurality of candidate predictive model network architectures by applying a different set of one or more model parameters, including a different lookback time window length parameter and a different window overlap parameter; determining a modified predictive model network architecture that is most suitable for solving the time series forecasting task of the current use case from among the plurality of candidate predictive model network architectures; 2. A computer-implemented system configured to execute

10. 10. The computer-implemented system of claim 9, wherein the predictive model network architecture includes one or more deep neural network models for time series forecasting selected from a convolutional neural network, a recurrent neural network, and a multi-layer perceptron neural network, or a combination thereof.

11. To select a predictive model network architecture for the current use case, the at least one processor:

10. The computer-implemented system of claim 9, further configured to search a library of existing predictive model network architectures that were previously generated for past use cases related to the current prediction context.

12. The at least one processor Obtaining characteristic features of the input time series data; Identifying similarities between the characteristic features of the input time series data and features of data used as input to an existing predictive model network architecture; selecting the existing predictive model network architecture configured for the time series forecasting task of the current use case based on the identified similarities; and The computer-implemented system of claim 9 , further configured to execute:

13. The at least one processor 11. The computer-implemented system of claim 10, further configured to accept, via a user interface, user constraints for performing the time series forecasting task of the current use case, the one or more user constraints comprising at least one or more of a specific predictive model network architecture, a model training time limit, a prediction accuracy, or an execution time ratio.

14. The at least one processor outputting the best-fit predictive model network architecture via the user interface; and presenting, via the user interface, a description of one or more dominant features in the best-fit predictive model network architecture; and The computer-implemented system of claim 13 further configured to execute:

15. The plurality of candidate predictive model network architectures to be trained include individuals, and the at least one processor: generating a plurality of individuals; For each individual, modifying each of the plurality of candidate predictive model network architectures by applying a different set of one or more model parameters while training in parallel; The computer-implemented system of claim 10 further configured to execute:

16. The at least one processor selecting a random set of individuals; determining the most fit individual from the random set; applying random mutations to one or more candidate predictive model network architectures of the determined fittest individuals; and 16. The computer-implemented system of claim 15, further configured to execute:

17. A computer-readable program that, when executed on a computer including at least one processor, causes the at least one processor to: receiving a prediction context related to a current use case; selecting an existing predictive model network architecture configured for a similar use case time series forecasting task based on the relevant forecasting context; and replicating the selected existing predictive model network architecture to generate a plurality of candidate predictive model network architectures; inputting time series data for each of the plurality of candidate predictive model network architectures; training each of the plurality of candidate predictive model network architectures in parallel using the input time series data; modifying each of the plurality of candidate predictive model network architectures by applying, while training in parallel, a different set of one or more model parameters, including a different lookback time window length parameter and a different window overlap parameter; determining a modified predictive model network architecture that is most suitable for solving the time series forecasting task of the current use case from among the plurality of candidate predictive model network architectures; A computer-readable program that causes the computer to execute the following:

18. to select a predictive model network architecture for the current use case, the at least one processor; 20. The computer-readable program of claim 17, further comprising: searching a library of existing predictive model network architectures that were previously generated for past use cases related to a current prediction context.

19. the at least one processor; Obtaining characteristic features of the input time series data; Identifying similarities between the characteristic features of the input time series data and features of data used as input to an existing predictive model network architecture; selecting the existing predictive model network architecture configured for the time series forecasting task of the current use case based on the identified similarities; and 18. The computer readable program of claim 17,

20. the at least one processor; outputting the best fitting predictive model network architecture via a user interface; and presenting, via the user interface, a description of one or more dominant features in the best-fit predictive model network architecture; and 18. The computer readable program of claim 17,

21. The plurality of candidate predictive model network architectures to be trained comprises individuals, and the at least one processor is configured to: generating a plurality of individuals; For each individual, modifying each of the plurality of candidate predictive model network architectures by applying a different set of one or more model parameters while training in parallel; 18. The computer readable program of claim 17,

22. the at least one processor; selecting a random set of individuals; determining the most fit individual from the random set; applying random mutations to one or more candidate predictive model network architectures of the determined fittest individuals; and 22. The computer readable program of claim 21,

Images