Computing device, computer-implemented method, and computer-readable storage medium for multivariate time series modeling and forecasting (Probabilistic nonlinear relationships and exogenous factors across multiple time series for improved multivariate time series modeling and forecasting)

Abstract

To provide a computing device, computer-implemented method and computer-readable storage medium for multivariate time series modeling and forecasting.SOLUTION: A computer-implemented method 600 of time series modeling and forecasting comprises: encoding a plurality of inputs of time series data 605; mapping the encoded multivariate time series data to a low-dimensional latent space 610; predicting next values in time of the encoded multivariate time series data in the low-dimensional latent space 615; mapping the predicted next values and a random noise back to an input space to provide a predictive distribution sample for a next time points of the multivariate time series data 620; and outputting one or more of time series forecasts based on the predictive distribution sample 625.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates generally to computer-implemented methods and systems for time series modeling, and more particularly to multivariate time series modeling and forecasting. [Background technology]

[0002] Modeling and forecasting a large number of time series and understanding their inter-series influences remains challenging.

[0003] For example, manual heuristic approaches lack the flexibility to capture cross-series influences, making such approaches unscalable. In demand forecasts with thousands to billions of product-location combinations, cross-product influences may occur.

[0004] They also lack the ability to capture underlying nonlinear relationships and influences over time, and there are problems with attempts to consider exogenous information and interactions with other factors.

[0005] The result is inappropriate and incorrect decisions based on flawed models, leading to increased inefficiencies, higher costs, wasted resources, missed opportunities, and poor quality products. Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, there is a need for a scalable, automated method to model nonlinear stochastic relationships between series and incorporate this into improved forecasting. [Means for solving the problem]

[0007] According to one embodiment, a computing device for time series modeling and forecasting includes a processor and a memory coupled to the processor. The memory stores instructions that cause the processor to perform operations including encoding input multivariate time series data and nonlinearly mapping the encoded multivariate time series data to a low-dimensional latent space. A next value in time of the encoded multivariate time series data in the low-dimensional latent space is predicted. The predicted next value and random noise are mapped back to an input space to provide a predictive distribution sample for a next time point of the multivariate time series data. One or more time series forecasts are output based on the predictive distribution sample. This improves the accuracy and time required for processing time series modeling and forecasting.

[0008] In one embodiment, the computing device is configured to train a neural network deep learning model to compute time series modeling and the one or more time series forecasts. The use of a neural network increases the efficiency of time series modeling and forecasting.

[0009] In one embodiment, the training of the deep learning model is unsupervised. The use of unsupervised training allows for broader pattern recognition and aids in the discovery of hidden patterns.

[0010] In one embodiment, the deep learning model is an end-to-end deep learning model trained using stochastic gradient descent, which makes the overall operation more efficient and the use of stochastic gradient descent can minimize input spatial prediction error.

[0011] In one embodiment, the end-to-end deep learning model includes an encoder neural network configured to encode inputs of multivariate time series data, a temporal predictor network configured to predict a next value in time from the encoded multivariate time series data received from the encoder neural network, and a decoder neural network configured to map the predicted next value from the temporal predictor network to an input space. The use of neural networks allows for more efficient operation and easier training.

[0012] In one embodiment, the decoder neural network is further configured to map a combination of random noise and latent space values ​​back to the input space, where random noise is used to increase pattern detection.

[0013] In one embodiment, the encoder neural network is further configured to encode extrinsic factor data for each series and time point of the input multivariate time series data before performing a nonlinear mapping of the encoded multivariate time series data to a low-dimensional latent space. The use of extrinsic factor data improves the accuracy of prediction by considering factors not seen in the time series data.

[0014] In one embodiment, the input multivariate time series data and the extrinsic factor data are arranged as a three-dimensional array with the third dimension corresponding to a feature of the extrinsic factor data.

[0015] In one embodiment, the encoder neural network is a temporal autoencoder, which improves temporal matrix factorization.

[0016] In one embodiment, the encoder neural network is a probabilistic temporal autoencoder, which has the continuous ability to model complex distributions of multivariate data via decoder mapping, and is a relatively simple structure that can be introduced into latent variables.

[0017] In one embodiment, the number of auto-encoded temporal patterns output by the temporal auto-encoder is less than the number of multivariate time series data input, a smaller number resulting in a shorter time to output a prediction.

[0018] According to one embodiment, a computer-implemented method for multivariate time series modeling and forecasting includes encoding a plurality of inputs of the multivariate time series data, mapping the encoded multivariate time series data into a low-dimensional latent space, predicting a next value in time of the encoded multivariate time series data in the low-dimensional latent space, and mapping the predicted next value and random noise back into an input space to provide a predictive distribution sample for a next time point of the multivariate time series data. One or more time series forecasts are output based on the predictive distribution sample. This improves the accuracy and time required for processing time series modeling and forecasting.

[0019] In one embodiment, the encoding of the plurality of multivariate time series data is performed by temporal autoencoding, which improves on temporal matrix factorization.

[0020] In one embodiment, the encoding of the plurality of multivariate time series data is performed by probabilistic temporal auto-encoding.

[0021] In one embodiment, the number of auto-encoded input multivariate time series data is greater than the number of auto-encoded temporal patterns output by the temporal auto-encoder, thereby improving processing speed.

[0022] In one embodiment, the mapping of the encoded multivariate time series data to a low-dimensional latent space is performed non-linearly. Using a non-linear mapping increases the detection of hidden patterns.

[0023] In one embodiment, a neural network deep learning model is trained to model the time series and compute the one or more time series predictions.

[0024] In one embodiment, an end-to-end deep learning model is provided, and the end-to-end deep learning model is trained using stochastic gradient descent, which can minimize a reconstruction error, a latent space prediction error, and an input space prediction error.

[0025] In one embodiment, the input multivariate time series data and the exogenous factor data are formed as a three-dimensional array with the third dimension corresponding to features of the exogenous factor data. The use of exogenous factor data improves the accuracy of prediction by considering factors not seen in the time series data.

[0026] According to one embodiment, a non-transitory computer-readable storage medium tangibly embodied with computer-readable program code having computer-readable instructions that, when executed, cause a computing device to perform a method for multivariate time series modeling and forecasting, the method comprising: encoding a plurality of inputs of multivariate time series data; mapping the encoded multivariate time series data into a low-dimensional latent space; predicting a next value in time of the encoded multivariate time series data in the low-dimensional latent space; mapping the predicted next value and random noise back into an input space to provide a predictive distribution sample for a next time point of the multivariate time series data; and outputting one or more time series forecasts based on the predictive distribution sample. This improves the accuracy and time required for processing time series modeling and forecasting.

[0027] These and other features will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.

[0028] The drawings are exemplary embodiments. They do not depict all embodiments. Other embodiments may be used in addition or instead. To save space or for a more effective explanation, details that may be obvious or unnecessary may be omitted. Some embodiments may be practiced with additional components or steps, or without all components or steps shown, or both. The same numerals appearing in different drawings refer to the same or similar components or steps. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 provides an architectural overview of a system for multivariate time series modeling and forecasting, consistent with an illustrative embodiment. [Figure 2] FIG. 2 illustrates an encoder neural network incorporating extrinsic factors per time series, consistent with an exemplary embodiment. [Figure 3] FIG. 3 illustrates a temporal autoencoder consistent with an exemplary embodiment. [Figure 4] FIG. 4 illustrates a probabilistic temporal autoencoder consistent with an exemplary embodiment. [Figure 5A] FIG. 5A shows dataset statistics and execution time per epoch, illustrating the improved functionality of the computer-implemented method of the present disclosure. [Figure 5B] FIG. 5B shows dataset statistics and execution time per epoch, illustrating the improved functionality of the computer-implemented method of the present disclosure. [Figure 6] FIG. 6 is a flowchart illustrating a computer-implemented method for time series modeling and forecasting consistent with illustrated embodiments. [Figure 7] FIG. 7 is a functional block diagram of a communications-enabled computer hardware platform consistent with an illustrative embodiment. [Figure 8] FIG. 8 illustrates an exemplary cloud computing environment, consistent with an exemplary embodiment. [Figure 9] FIG. 9 illustrates a set of functional abstraction layers provided by a cloud computing environment, consistent with an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0030] (overview) In the following detailed description, numerous specific details are set forth by way of example to provide a thorough understanding of the relevant teachings. However, it should be understood that the present teachings may be practiced without such details. In other instances, well-known methods, procedures, components, or circuits, or combinations thereof, have been described at a relatively high level, without detail, in order to avoid unnecessarily obscuring aspects of the present teachings.

[0031] As used in some exemplary embodiments herein, the term "latent space" refers to an abstract multidimensional space containing feature values ​​that are not directly interpreted, but which are used to encode meaningful internal representations of externally observed phenomena, and the term "low-dimensional latent space" refers to a reduction in the original spectral dimensionality to increase the efficiency of search.

[0032] The term "input space" is understood to mean all possible inputs in machine learning. For example, in an exemplary embodiment, random noise samples are decoded to provide a predictive distribution. A decoder can be configured to map the random noise back into the input space.

[0033] Additionally, the term "stochastic gradient descent" generally refers to a method for reducing errors by approximating the gradient of training samples. In some exemplary embodiments, stochastic gradient descent can be used to minimize reconstruction error, potential spatial prediction error, and input spatial prediction error.

[0034] The computer-implemented methods and devices of the present disclosure provide improvements in the field of time series modeling and forecasting, the increased accuracy of which has led to increased efficiency in a variety of fields, including, but not limited to, health (e.g., manufacturing and distribution of medicines, vaccines, etc.), food, waste management, and communications (network operations, internet traffic, data streaming, etc.), to name just a few.

[0035] Furthermore, the computer-implemented methods and devices of the present disclosure provide improved efficiency of computer operations. Through the teachings herein, technical improvements result in reduced amount and power of processing requirements. For example, improved prediction accuracy requires fewer iterations, thereby freeing up computer resources. Time savings are also realized using the teachings of the present disclosure.

[0036] (Architecture example) FIG. 1 provides an overview of an architecture 100 for multivariate time series modeling and forecasting consistent with an exemplary embodiment. Time series data 101, in this exemplary embodiment, is multivariate time series data. Multivariate data is data for which analysis is based on two or more variables per observation. Multivariate time series data 105 is a collection of multiple variables at a subsequent time point. The multiple multivariate time series data 105 are shown to represent some of the various data patterns. The multivariate data is input to an encoder 111. The encoder 111 is configured to encode the multivariate time series data into a smaller number of shared / global underlying temporal patterns 113 and a nonlinear combination of the cleaned and denoised input time series data. The encoder also performs a nonlinear mapping of the encoded multivariate time series data to a low-dimensional latent space. As noted above, the term "low-dimensional latent space" refers to a reduction in the original spectral dimensionality to enhance search efficiency.

[0037] The temporal model 115 receives the encoded multivariate time series data and predicts the next value of the encoded multivariate time series data in a low-dimensional space. The predicted next value is provided by the temporal model 115 as a prediction 114 in a latent space. In this illustrated embodiment, the temporal model 115 is a recurrent neural network (RNN) and may be referred to as a temporal predictor network. However, the temporal model 115 of the present disclosure is not limited to an RNN.

[0038] The decoder 117 receives the predicted next value prediction 114 and is configured to map the predicted next value from the time model to the input space. The decoder 117 receives noise from the random noise generator 119 combined with the latent sequence values ​​and the predicted values ​​114 and generates random noise samples in the latent space, for example, by adding the random noise directly to the values ​​and predictions. The predictions can include any characteristics of the joint distribution, including the mean or median, variance, various quantiles, etc. The decoder 117 decodes the random noise samples to provide sampled prediction distributions 123, 124 across the time series. Also shown are the reconstructed sequence mean input sequence 121 (zero noise input) and the reconstructed mean prediction 122 (zero noise input). The predictions are output to a storage device 125 and then to a decision / optimization and planning system 127. The prediction output can be used as desired by the user. Thus, from the architecture shown in FIG. 1, it can be seen that time series modeling and prediction provides outputs that can be used by algorithms in other systems.

[0039] For example, the latent space includes a latent / global sequence and a prediction. As described above, random noise is added directly to the latent space values / predictions. However, there are other ways to combine random noise with the latent space values / predictions. For example, if the prediction includes both a mean and standard deviation prediction, the random noise can be transformed to have an output standard deviation in the latent space, e.g., scaling the prediction by the standard deviation output before adding it to the mean output.

[0040] 2 illustrates an encoder neural network 200 incorporating extrinsic factors per time series, consistent with an exemplary embodiment. The encoder 211 in this exemplary embodiment is configured to receive extrinsic factor data 203 per series along with time series data 201 (e.g., specific features per series and time point), where the inputs are arranged as tensors, and the extrinsic factor data is added along with the time series data as another dimension, such as to form a three-dimensional array or tensor, where the additional dimension corresponds to the extrinsic features from the extrinsic data for each individual time series. The temporal model 215 operates as described with respect to FIG. 1.

[0041] Time series data 205 is shown as being supplemented with weather features 209 for each series and time point. The feature series and time points may be any feature that needs to be predicted or may affect the prediction of the target series. For example, if the time series data 205 is Internet traffic and a major sporting event is about to take place, the sporting event 207 may be the characteristics and time of the game. Weather features 209 may affect the sporting event, causing game delays, etc., and Internet traffic for viewers streaming the event may allow for predicting Internet traffic. For example, a telecommunications company may increase network capacity and, if possible, enable an appropriate number of network servers to handle the traffic.

[0042] Figure 3 illustrates a temporal autoencoder 300 consistent with an exemplary embodiment. An encoder 311 and a decoder 317 are shown. The temporal autoencoder 300 can be embodied as a multivariate temporal autoencoder, configured to discover latent features corresponding to a latent time series and represent them as hidden state vectors per time point. The latent time series are modeled as having explicit temporal patterns described by a latent time series model, which models how the time series progress over time and how they interact in a potentially nonlinear manner within the latent space. One such temporal model is shown in the equation in the figure, where future time series values ​​(x for a particular t) of the latent time series are a linear function, i.e., a weighted sum, of previous latent time series values. More complex nonlinear temporal models have also been proposed, such as recurrent neural networks (RNNs) and temporal convolutional neural networks (TCNs), where future latent values ​​are typically a function of previous latent values. Thus, the entire process of a temporal autoencoder can be modeled as a single end-to-end sequence of operations or functions: mapping an input space time series to a latent space, transforming within the latent space, and then mapping back to the input space. Each step can be modeled with an arbitrary function, such as a neural network, of a variety of different architectural types. Thus, in the same way that temporal models, including neural nets like RNNs and TCNs, are trained on time series or sequence data, this entire end-to-end model, including the entire flow, is trained in the same way, using subsequences or batches of time series, and all components—the encoder, decoder, and latent temporal model—are simultaneously optimized using stochastic gradient descent and backpropagation, with gradients calculated at each update step.

[0043] FIG. 4 illustrates a probabilistic temporal autoencoder 400 consistent with an exemplary embodiment. An encoder 411 and a decoder 417 are shown. One of the problems in time series prediction is how future values ​​are probabilistically modeled. According to this exemplary embodiment, high-dimensional data is encoded into a low-dimensional embedding, and a latent space probability model is based on the low-dimensional embedding. Prediction samples can be obtained by sampling from a latent distribution and transforming the prediction samples through a decoder to obtain probabilistic samples in a (more complex) input space. If the encoder is complex enough to capture nonlinear correlations between series and the decoder is complex enough to map simple distributions to more complex distributions (similar to the idea of ​​inverse transform sampling commonly used in statistics), a relatively simple probability structure can be introduced into the latent variables, with the ongoing ability to model complex distributions of multivariate data via decoder mapping.

[0044] Continuing with Figure 4, we can use variational operations similar to the idea of ​​a variational autoencoder (VAE) to draw samples in the latent space. More specifically, the latent space variance σ 2 is fixed to '1' to simplify the modeling and avoid overfitting in Equation 1 below. TIFF0007751936000001.tif7115Where "P" is the probability and X l+1 is the next value in the multivariate latent space-time series, and "N" is the probability of the next value given the previous observations in the latent space.

[0045] Here, without loss of generality, the probability P follows a normal distribution, and the output of the latent time model is the next value X l+1The latent space is assumed to have a mean given by the predicted mean of the latent time model and a standard deviation that is either given by another output of the latent time model or fixed to a constant value (e.g., "1") as described above. In this way, samples can be made by drawing random samples from a given probability distribution in the latent space given the latent time model output. These transformed samples in the input space correspond to samples from the joint distribution of future values ​​across the time series as the decoder and latent space model are fitted to the observed data. From these joint distribution samples, any properties of the distribution can be provided as different types of predictions. For example, these properties can include the mean or median, variance, various quantiles, etc. These properties can be used to provide various types of key predictions for different applications, such as the median and predictions of the 5th and 95th percentiles to provide standard prediction intervals.

[0046] 5A and 5B show the execution time per epoch and a comparison with other algorithms, demonstrating the improved functionality of the computer-implemented method of the present disclosure. Figure 5A shows that the large version of the wiki 515 took only 1.5 times longer per epoch than the small version of the wiki 510. However, since the large wiki had 57 times as many sequences as the small wiki, the computer-implemented method is particularly improved with large data, providing significant savings in time and resources.

[0047] Figure 5B provides a comparison of different algorithms with the computer-implemented method of the present disclosure (identified as "TLAE"). TLAE550 uses a smaller latent space size than DeepGLO560 and outperformed all compared global factorization models. TLAE did not use exogenous predictors such as day of the week or time of day or local modeling, yet it outperformed all other methods on most datasets.

[0048] (Process example) With the above overview of an exemplary architecture, it may be useful to now consider a high-level discussion of an exemplary process. To that end, in conjunction with FIGS. 1-5, FIG. 6 illustrates a flowchart 600 illustrating a computer-implemented method for time series modeling and forecasting consistent with an exemplary embodiment. Process 600 is illustrated as a collection of blocks in a logical flowchart that represent sequences of operations that can be performed in hardware, software, or a combination thereof. In the software context, the blocks represent computer-executable instructions that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, etc. that perform a function or implement an abstract data type. In each process, the order in which operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order, executed in parallel, or both, to perform the process.

[0049] 6, multiple inputs of time series data are encoded in operation 605. The time series data can be virtually any type of data being tracked over time, including, but not limited to, sensor data from electronic devices (both stationary and mobile), electronic vehicles and vehicle traffic flows, production information, product sales, network bottlenecks, etc.

[0050] In operation 610, the encoded multivariate time series data is mapped to a low-dimensional latent space. A low-dimensional latent space refers to a space in which low-dimensional representations are drawn. Machine learning utilizes low-dimensional latent spaces for many reasons, including, but not limited to, predicting missing variables.

[0051] In operation 615, there is a prediction of the next time value of the encoded multivariate time series data in the low-dimensional latent space. Through successive iterations, the global time series patterns can be accurately captured, the latent variables in the low-dimensional latent space can have their own local properties, and the output predicted samples can be calculated from the predicted latent samples.

[0052] In operation 620, the predicted next value and random noise are mapped back to the input space to provide a sample predictive distribution for the next time point of the multivariate time series data. Noise increases the difficulty of identifying patterns, and random noise can be used to aid decoding and obtain distributions across the series and predictions.

[0053] In operation 625, one or more of the time series forecasts based on the predicted distribution samples are output. The output may be stored and / or provided to a decision optimization and planning system that operates its own algorithms based in part on the provided forecast sampling.

[0054] 7 provides a functional block diagram of a computer hardware platform 700. In particular, FIG. 7 illustrates a specially configured network or host computer platform 700 that can be used to implement the method illustrated in FIG.

[0055] The computer platform 700 may include a central processing unit (CPU) 704, a hard disk drive (HDD) 706, a random access memory (RAM) and / or read-only memory (ROM) 708, a keyboard 710, a mouse 712, a display 714, and a communication interface 716, which are connected to the system bus 702. The HDD 706 may include a data store.

[0056] In one embodiment, HDD 706 has functionality that includes storing programs that can perform various processes in the manner described herein, such as multivariate time series modeling and forecasting module 720. The multivariate time series modeling and forecasting module is an end-to-end deep learning model according to certain exemplary embodiments described herein. The end-to-end deep learning model can be trained using stochastic gradient descent, which can be based on training samples 750.

[0057] The encoder module 725 is configured to encode the multivariate time series input. The encoder module 725 may be embodied as a neural network. The encoder module 725 may also be configured to receive extrinsic factor data for each series 203 (see FIG. 2) along with the time series data 201 (e.g., specific features for each series and time point). If the inputs are arranged as tensors, the extrinsic factor data is added along with the time series data as another dimension (e.g., forming a three-dimensional array), with the additional dimension corresponding to the extrinsic features from the extrinsic data for each individual time series. The encoder module 725 may be configured to improve efficiency or enforce sparsity across the extrinsic factors using an attention model. The attention model takes all or some of the inputs / time series and determines the extrinsic factors, including how to weight them, potentially multiplying them by zero or excluding certain factor inputs on a case-by-case basis.

[0058] The nonlinear combination of input time series data is cleaned and denoised by the encoder module 725. The encoder module 725 outputs a smaller number of shared / global patterns than the input time series data. Fewer shared / global patterns means fewer patterns for the decoder module 740 to decode, thus increasing efficiency of operation. The speed of time series modeling and prediction is also increased by the encoder outputting a smaller number of shared / global patterns for the decoder to process.

[0059] The temporal predictor 730 is configured to predict a next value from the encoded multivariate time series data received from the encoder module 725. As described above, if the time series data includes exogenous factor data for each series (e.g., a three-dimensional array), the temporal predictor 730 is configured to predict a next value based on the encoded time series data and the exogenous factor data. The temporal predictor 730 is configured to provide a prediction in the latent space of the temporal model. The decoder module 740 is configured to map the predicted next value from the temporal predictor 730 to the input space. The random noise generator 745 adds random noise to the decoder module 740. The random noise samples are decoded to provide a distribution across the time series and the prediction.

[0060] (Example of a cloud platform) As noted above, the environment and functionality associated with the ecological optimization method may include a cloud. While this disclosure includes detailed descriptions of cloud computing, implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the present invention may be practiced in conjunction with any other type of computing environment now known or developed in the future.

[0061] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five characteristics, at least three service models, and at least four implementation models. The characteristics are as follows:

[0062] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time or network storage, automatically as needed, without the need for human interaction with the service provider.

[0063] Broad network access: Computing power is available over the network and can be accessed through standard mechanisms, facilitating use by heterogeneous thin or thick client platforms (e.g., cell phones, laptops, PDAs).

[0064] Resource Pooling: Computing resources from a provider are pooled and offered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated based on demand. Consumers generally have no control or knowledge of the exact location of the resources they are provided with, so there is a sense of location independence. However, consumers may be able to identify their location at a higher level of abstraction (e.g., country, state, data center).

[0065] Rapid Elasticity: Computing capacity can be provisioned quickly and elastically, sometimes automatically, to instantly scale out and quickly released to instantly scale in. To the consumer, the computing power available to provision often appears unlimited, and can be purchased at any time and in any quantity.

[0066] Metered Services: Cloud systems leverage measurement capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user accounts) to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of utilized services. The service model is as follows:

[0067] Software as a Service (SaaS): The functionality offered to the consumer is the availability of a provider's applications running on a cloud infrastructure. The applications can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., webmail). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, except for limited user-specific application configuration settings.

[0068] Platform as a Service (PaaS): The capability offered to consumers is to deploy applications they create or acquire onto a cloud infrastructure using programming languages ​​and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of their hosting environment.

[0069] Infrastructure as a Service (IaaS): The functionality provided to consumers is the provisioning of processors, storage, networking, and other basic computing resources on which the consumer can deploy and run any software, including operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, and deployed applications, and in some cases partial control over some network components (e.g., host firewalls). The deployment model is as follows:

[0070] Private Cloud: This cloud infrastructure is dedicated to a specific organization and can be managed by that organization or a third party, and can exist on-premise or off-premise.

[0071] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common concerns (e.g., mission, security requirements, policies, and compliance). This cloud infrastructure can be managed by those organizations or a third party and can exist on-premises or off-premises.

[0072] Public cloud: This cloud infrastructure is available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0073] Hybrid cloud: This cloud infrastructure combines two or more cloud models (private, community, or public), each of which retains its inherent nuances but is bound by standards or specific technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0074] A cloud computing environment is a service-oriented environment that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0075] FIG. 8 illustrates an exemplary cloud computing environment 800. As illustrated, the cloud computing environment 800 includes one or more cloud computing nodes 810, with which local computing devices used by cloud consumers (e.g., PDA or cell phone 854A, desktop computer 854B, laptop computer 854C, or automobile computer system 854N, or combinations thereof) can communicate. The nodes 810 can communicate with each other. The nodes 810 can be physically or virtually grouped (not shown) in one or more networks, such as the private, community, public, or hybrid clouds described above, or combinations thereof. This enables the cloud computing environment 850 to provide infrastructure, platform, or software as a service, or combinations thereof, for which cloud consumers are not required to maintain resources on their local computing devices. It should be understood that the types of computing devices 854A-N illustrated in FIG. 8 are merely exemplary, and that the computing nodes 810 and the cloud computing environment 850 can communicate with any type of electronic device via any type of network or network-addressable connection (e.g., using a web browser), or both.

[0076] A set of functional abstraction layers provided by the cloud computing environment 800 (FIG. 8) is now shown in FIG. 9. It should be understood in advance that the components, layers, and functions shown in FIG. 9 are merely exemplary, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0077] Hardware and software layer 960 includes hardware and software components. Examples of hardware components include mainframe 961, reduced instruction set computer (RISC) architecture-based server 962, server 963, blade server 964, storage device 965, and network and network components 966. In some embodiments, software components include network application server software 967 and database software 968.

[0078] The virtualization layer 970 provides an abstraction layer from which the following virtual entities can be provided, for example: virtual servers 971, virtual storage 972, virtual networks including virtual private networks 973, virtual applications and operating systems 974, and virtual clients 975.

[0079] As an example, the management layer 980 may provide the following functionality: Resource provisioning 981 enables dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 982 enables cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. As an example, these resources may include application software licenses. Security enables identification and verification of cloud consumers and tasks, as well as protection for data and other resources. User portal 983 provides consumers and system administrators with access to the cloud computing environment. Service level management 984 enables allocation and management of cloud computing resources so that requested service levels are met. Service level agreement (SLA) planning and fulfillment 985 enables advance arrangement and procurement of anticipated future cloud computing resources required in accordance with SLAs.

[0080] The Workloads layer 990 provides examples of functionality available to a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include mapping and navigation 991, software development and lifecycle management 992, virtual classroom instruction delivery 993, data analytics processing 994, transaction processing 995, and, as described above, a time series and forecasting module 996 for performing multivariate time series modeling and forecasting.

[0081] (Conclusion) The description of various embodiments of the present teachings is presented for purposes of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. It will be apparent to those skilled in the art that many modifications and variations are possible without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to explain the principles of the embodiments, practical applications or technical improvements to technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0082] While the above describes what is considered to be the best mode and / or alternative embodiments, it is understood that various modifications may be made therein, that the subject matter disclosed herein may be embodied in a variety of forms and embodiments, and that the teachings may be applied in numerous applications, only some of which are described herein. It is intended that the following claims claim all applications, modifications, and variations that fall within the true scope of the present teachings.

[0083] The components, steps, features, objects, values, and advantages described herein are merely exemplary. Neither they nor the discussion thereof is intended to limit the scope of protection. While various advantages have been discussed herein, it will be understood that not all embodiments necessarily include all advantages. Unless otherwise specified, all measurements, values, ratings, positions, dimensions, sizes, and other specifications described herein, including those set forth in the following claims, are approximate and not exact. They are intended to have a reasonable range consistent with the functions to which they relate and that which is customary in the technical field to which they pertain.

[0084] Numerous other embodiments are contemplated, including embodiments having fewer, additional, or different, or combinations of, components, steps, features, objects, values, and advantages, including embodiments in which components and / or steps are arranged and / or ordered differently.

[0085] The flowcharts and diagrams in the figures herein illustrate the architecture, functionality, and operation of possible implementations according to various embodiments of the present disclosure.

[0086] While the foregoing has been described in connection with exemplary embodiments, it is understood that the term "exemplary" means merely an example, not best or optimal. Except as noted above, nothing described or illustrated is intended to, or should be construed to, induce the public to dedicate any component, step, feature, object, value, advantage, or equivalent, whether or not claimed.

[0087] Terms and phrases used herein are understood to have the ordinary meanings given to such terms and phrases in relation to their respective fields of inquiry and study, unless a specific meaning is otherwise stated herein. Relationship terms such as first and second, etc., may be used merely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between such entities or actions. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements may not include only those elements, but may include other elements not expressly listed or inherent in such process, method, article, or apparatus. An element preceded by the singular form "a" or "an" does not, without further constraints, exclude the presence of additional identical elements in a process, method, article, or apparatus that includes that element.

[0088] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, in the foregoing Detailed Description, it will be appreciated that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Accordingly, the following claims are incorporated herein, with each claim standing on its own as separately claimed subject matter.

Claims

1. 1. A computing device for time series modeling and forecasting, comprising: a processor; a memory coupled to the processor, the memory providing to the processor: encoding input multivariate time series data using an encoder neural network and nonlinearly mapping the encoded multivariate time series data into a low-dimensional latent space; predicting a next value in time of the encoded multivariate time series data in the low-dimensional latent space with a temporal predictor network; mapping the predicted next value and random noise back to an input space with a decoder neural network to provide a sample predictive distribution of the next time point of the multivariate time series data; outputting one or more time series predictions based on the predictive distribution samples; A computing device storing instructions for performing operations including:

2. The instructions cause the processor to: training a neural network deep learning model to compute time series modeling and one or more time series forecasts; The computing device of claim 1 , further comprising:

3. The computing device of claim 2 , wherein the training of the deep learning model is unsupervised.

4. The computing device of claim 2 , wherein the deep learning model comprises an end-to-end deep learning model trained using stochastic gradient descent.

5. The end-to-end deep learning model the encoder neural network configured to encode an input of multivariate time series data; the temporal predictor network configured to predict a next value in time from the encoded multivariate time series data received from the encoder neural network; the decoder neural network configured to map the predicted next values ​​from the temporal predictor network to an input space; The computing device of claim 4 further comprising:

6. a noise generator configured to generate random noise input to the decoder neural network; the decoder neural network is further configured to map the combination of the random noise and latent space values ​​back to the input space. The computing device of claim 5 .

7. 6. The computing device of claim 5, wherein the encoder neural network is further configured to encode exogenous factor data for each series and time point of the input multivariate time series data before performing a nonlinear mapping of the encoded multivariate time series data to a low-dimensional latent space.

8. The computing device of claim 7 , wherein the input multivariate time series data and the exogenous factor data are arranged as a three-dimensional array with a third dimension corresponding to a feature of the exogenous factor data.

9. The computing device of claim 5 , wherein the encoder neural network comprises a temporal autoencoder.

10. The computing device of claim 9 , wherein the encoder neural network comprises a probabilistic temporal autoencoder.

11. The computing device of claim 10 , wherein the number of auto-encoded temporal patterns output by the temporal auto-encoder is less than the number of input multivariate time series data.

12. 1. A computer-implemented method for multivariate time series modeling and forecasting, the computer-implemented method comprising: Encoding multiple inputs of multivariate time series data with an encoder neural network; Mapping the encoded multivariate time series data into a low-dimensional latent space by the encoder neural network; predicting a next value in time of the encoded multivariate time series data in the low-dimensional latent space with a temporal predictor network; mapping the predicted next value and random noise back to an input space with a decoder neural network to provide a sample predictive distribution of the next time point of the multivariate time series data; outputting one or more time series predictions based on the predictive distribution samples; 11. A computer-implemented method comprising:

13. The computer-implemented method of claim 12, wherein the encoding of the plurality of multivariate time series data is performed by temporal auto-encoding.

14. The computer-implemented method of claim 12, wherein the encoding of the plurality of multivariate time series data is performed by probabilistic time auto-encoding.

15. 14. The computer-implemented method of claim 13, wherein the number of auto-encoded input multivariate time series data is greater than the number of auto-encoded temporal patterns output by the temporal auto-encoder.

16. 14. The computer-implemented method of claim 13, wherein the mapping of the encoded multivariate time series data to a low-dimensional latent space comprises a non-linear mapping.

17. 14. The computer-implemented method of claim 13, further comprising training a neural network deep learning model to compute time series modeling and the one or more time series predictions.

18. 14. The computer-implemented method of claim 13, further comprising providing an end-to-end deep learning model and training the end-to-end deep learning model using stochastic gradient descent.

19. 14. The computer-implemented method of claim 13, further comprising forming the input multivariate time series data and exogenous factor data as a three-dimensional array, with a third dimension corresponding to a feature of the exogenous factor data.

20. 1. A non-transitory computer-readable storage medium tangibly embodied with computer-readable program code having computer-readable instructions that, when executed, cause a computing device to perform a method for multivariate time series modeling and forecasting, the method comprising: Encoding multiple inputs of multivariate time series data with an encoder neural network; Mapping the encoded multivariate time series data into a low-dimensional latent space by the encoder neural network; predicting a next value in time of the encoded multivariate time series data in the low-dimensional latent space with a temporal predictor network; mapping the predicted next value and random noise back to an input space with a decoder neural network to provide a sample predictive distribution of the next time point of the multivariate time series data; outputting one or more time series predictions based on the predictive distribution samples; 1. A computer-readable storage medium comprising:

Images