System and method for determining dependencies in multivariate time series data

Abstract

A system and method for multivariate time series data analysis comprising a memory storing computer-executable components, and a processor operatively coupled to the memory to execute the computer-executable components stored in the memory, the computer-executable components including a time series analysis component that generates a machine learning model that discovers dependencies among multivariate time series data using an attention mechanism controlled by an uncertainty measure.

Description

[Technical field]

[0001] The present invention relates to determining dependencies from multivariate time series data, and more particularly to determining dependencies in multivariate time series data along with reliability values ​​using deep learning neural networks. [Background technology]

[0002] Deep learning neural networks have become the benchmark model for multiple machine learning problems, such as neural machine translation, image classification, and subtitle generation. However, the use of deep learning neural networks in time series analysis remains challenging and has so far focused on the traditional task of forecasting time series data. For many practical time series applications, discovering dependencies between multivariate time series data along with their reliability can help explain correlated and / or causal events behind the observed data and provide more insight into the time series variations, thus assisting users in making better decisions. Such applications can include, for example, cloud service management, financial investment, and medical safety.

[0003] The past success of deep learning is attributed not only to its ability to generate complex and informative features by applying multiple steps of nonlinear data transformations, but also to the adoption of an attention mechanism that can act as a memory access mechanism by generating large attention coefficients for input features relevant to the learning task. However, attention mechanisms can be associated with a large amount of instability since the learning process is usually unsupervised, with no explicit ground truth regarding attention available during the training phase. Furthermore, time series data can be highly noisy, volatile, and time-varying, and therefore modeling time series data using traditional neural networks can lead to overconfidence in dependencies and predicted values, as well as a lack of precision in assessing the uncertainty of the data. Summary of the Invention

[0004] The following presents a summary to provide a basic understanding of one or more embodiments of the present invention. This summary is not intended to identify key or critical elements, nor is it intended to delineate the scope of particular embodiments or claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, a system, computer-implemented method, apparatus, and / or computer program product that can determine dependencies in multivariate time series data along with certainty values ​​are described.

[0005] According to one aspect of the present invention, a system is provided. The system comprises a memory storing computer-executable components. The system also comprises a processor operatively coupled to the memory and executing the computer-executable components stored in the memory. The computer-executable components include a time series analysis component that generates a machine learning model that uses an attention mechanism based on an uncertainty measure to discover dependencies between multivariate time series data. An advantage of such a system can be that it enables highly accurate forecasts based primarily on unsupervised machine learning training.

[0006] In some examples, the system can further comprise an encoder component capable of encoding the multivariate time series data by matching different sampling rates of the data across a set of constructed multivariate time series data. The encoding can make the multivariate time series data scalable and parallelizable. The system can also comprise an attention decoder component capable of regularizing attention across the encoded multivariate time series data based on an uncertainty measure using an attention network layer of a machine learning model. An advantage of such a system can be the encoding of the multivariate time series data in a manner that can make the data scalable and parallelizable despite long data sequences.

[0007] According to another aspect, a computer-implemented method is provided that includes generating, by a system operatively coupled to a processor, a machine learning model that uses an attention mechanism based on an uncertainty measure to discover dependencies among multivariate time series data. An advantage of such a computer-implemented method can be the implantation of an attention mechanism that can regularize deep neural networks away from learning variable or random attention.

[0008] In some examples, the computer-implemented method may further include regularizing attention among the encoded multivariate time series data based on the uncertainty measure using an attention network layer of the machine learning model. The computer-implemented method may also include generating, by the system, an uncertainty quantification for the dependencies via Monte Carlo sampling performed on the attention network layer. The multivariate time series data may characterize a measurable property. Furthermore, the computer-implemented method may include predicting, by the system, a value of the measurable property based on the discovered dependencies and the uncertainty quantification. An advantage of such a computer-implemented method may be insight into correlations and / or causal relationships between multiple variables that characterize an observable system operating state.

[0009] According to another aspect, a computer program product for analyzing multivariate time series data is provided. The computer program product comprises a computer readable storage medium having program instructions embodied thereon. The program instructions are executable by a processor to cause the processor to generate a machine learning model that uses an attention mechanism based on an uncertainty measure to discover dependencies between the multivariate time series data. An advantage of such a computer program product can be exploitation of the convergence rate of the machine learning model in narrowing down the search space for attention patterns in the data.

[0010] In some examples, the program instructions may further cause the processor to encode the multivariate time series data by matching different data sampling rates across a set of multivariate time series data constructed by the processor, the encoding making the multivariate time series data scalable and parallelizable. Additionally, the program instructions may further cause the processor to regularize attention among the encoded multivariate time series data based on an uncertainty measure using an attention network layer of the machine learning model. Additionally, the program instructions may further cause the processor to penalize attention configurations of the encoded multivariate time series data that result in an uncertainty measure being less than a defined threshold by the processor. An advantage of such a computer program product may be the use of Bayesian theory to learn a probability distribution across all of the neural weights of the neural network model.

[0011] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the U.S. Patent Office upon request and payment of the necessary fee. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates a block diagram of an example non-limiting system capable of discovering dependencies in multivariate time series data in accordance with one or more embodiments described herein. [Diagram 2] FIG. 1 illustrates a block diagram of an example non-limiting system that can encode multivariate time series data to make the data scalable and parallelizable in accordance with one or more embodiments described herein. [Diagram 3]FIG. 1 illustrates a block diagram of an example, non-limiting system that can aggregate and distribute encoded multivariate time series data across one or more attention network layers of a machine learning model in accordance with one or more embodiments described herein. [Figure 4] FIG. 1 illustrates a block diagram of an example, non-limiting system capable of discovering dependencies from multivariate time series data and / or quantifying the amount of certainty associated with dependencies in accordance with one or more embodiments described herein. [Diagram 5] FIG. 1 illustrates an example non-limiting learning objective function that can be implemented by one or more machine learning models to discover dependencies from multivariate time series data, in accordance with one or more embodiments described herein. [Figure 6] FIG. 1 illustrates a flow diagram of an example, non-limiting method that can be implemented by one or more autonomous computer systems to discover one or more dependencies between multivariate time series data, in accordance with one or more embodiments described herein. [Figure 7] FIG. 1 illustrates example, non-limiting graphs that may indicate the effectiveness of one or more machine learning techniques in discovering dependencies in multivariate time series data, forecasting time series data, and / or quantifying relevant amounts of uncertainty, in accordance with one or more embodiments described herein. [Figure 8] FIG. 1 illustrates exemplary non-limiting charts and graphs that can demonstrate the effectiveness of one or more systems that can discover dependencies in multivariate time series data, in accordance with one or more embodiments described herein. [Figure 9] FIG. 1 illustrates a flow diagram of an example non-limiting computer-implemented methodology that can facilitate discovering dependencies in multivariate time series data in accordance with one or more embodiments described herein. [Figure 10]FIG. 1 illustrates a flow diagram of an example non-limiting computer-implemented methodology that can facilitate discovering dependencies in multivariate time series data in accordance with one or more embodiments described herein. [Figure 11] FIG. 1 illustrates a cloud computing environment in accordance with one or more embodiments described herein. [Figure 12] FIG. 1 illustrates an abstract model layer in accordance with one or more embodiments described herein. [Figure 13] FIG. 1 is a block diagram illustrating an exemplary non-limiting operating environment that can facilitate one or more embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following detailed description is merely exemplary and is not intended to limit the embodiments or the application and / or uses of the embodiments, nor is it intended to be bound by any express or implied information presented in the preceding "Background" or "Summary" sections, or in the "Description of the Preferred Embodiments" section.

[0014] One or more embodiments are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more embodiments. However, it will be apparent that, in various instances, one or more embodiments may be practiced without these specific details.

[0015] Given the problems associated with other implementations of time series data analysis, the present disclosure can be implemented to generate solutions to one or more of these problems by employing deep neural network models that can learn to discover dependencies between stable and reliable time series while providing a confidence value associated with the generated output. One or more embodiments described herein can advantageously employ one or more attention mechanisms that implement one or more Shannon entropy-based penalties, and can leverage the convergence rate of the machine learning model as the model narrows down the search space for attention patterns in the time series data. Furthermore, unlike traditional deep learning models that employ point estimation techniques, various embodiments described herein can train an infinite set of deep learning models that include weights that can be characterized by one or more probability distributions.

[0016] Various embodiments of the present invention may be directed to a computer processing system, computer-implemented method, apparatus, or computer program product, or combination thereof, that facilitates efficient, effective, and autonomous time series data analysis (e.g., without direct human guidance). For example, one or more embodiments described herein may discover dependencies (e.g., interdependencies) in one or more time series data, generate confidence values ​​regarding the discovered dependencies, and predict future time series data based on the dependencies or the associated confidence values, or both. For example, one or more embodiments described herein may consider a deep learning neural network that may employ one or more attention mechanisms, where the attention mechanism penalizes the learning objective function based on a measure of uncertainty (e.g., Shannon entropy) that is calculated directly from the attention coefficients during the training process. Various embodiments may thereby regularize the deep neural network away from learning variable or random attention.

[0017] The computer processing system, computer implemented method, apparatus, and / or computer program product employs hardware and / or software to solve problems that are highly technical in nature, not abstract, and cannot be performed as a series of mental activities by a human (e.g., unsupervised discovery of dependencies in multivariate time series). For example, one or more humans cannot discover interdependencies in vast amounts of time series data with the efficiency achieved by various embodiments described herein. For example, humans cannot easily detect and / or analyze fine pattern recognition across multiple variables in time series data that may contain tens, hundreds, or even thousands of time frames. Additionally, various embodiments described herein may employ machine learning techniques that may enable a level of data confidentiality that cannot be replicated by human capabilities.

[0018] Additionally, one or more embodiments described herein may constitute a technical improvement over traditional time series data analysis by generating confidence values ​​associated with discovered time series data dependencies to provide further explanation for correlated / causal events behind the observed data and / or time series fluctuations. Additionally, various embodiments described herein may represent a technical improvement over traditional time series analysis by utilizing Bayesian theorem to enable deep learning neural networks to learn probability distributions over all neural weights, thereby enhancing the representation of model confidence.

[0019] Additionally, one or more embodiments described herein may have practical applications by providing insight into the operation and / or maintenance of one or more mechanical or computer systems, or mechanical or computer functions, or combinations thereof. For example, various embodiments described herein may discover dependencies in time series data, which may characterize correlations and / or causal relationships between variables such that effects from the manipulation of one variable on another may be predicted based on observed experience. One or more embodiments described herein may control one or more attention network layers of a deep learning neural network to employ an attention mechanism that penalizes attention configurations that result in high and unstable measures of uncertainty (e.g., Shannon entropy values). One or more embodiments may thereby leverage the convergence rate of machine learning models while narrowing down the search space for attention patterns in the data.

[0020] As used herein, the term "machine learning" can refer to the application of artificial intelligence techniques to automatically and / or autonomously learn and / or improve from experience (e.g., training data) without explicit programming of the learned and / or improved knowledge. For example, machine learning operations can utilize one or more algorithms to facilitate supervised and / or unsupervised learning to perform operations such as classification, regression, or clustering, or a combination thereof. The performance of machine learning operations can be facilitated by one or more artificial intelligence models trained on one or more datasets according to one or more model configuration settings.

[0021] As used herein, the term "machine learning model" may refer to a computer model that may be used to facilitate one or more machine learning tasks, where the computer model may simulate multiple interconnected processing units that may resemble abstract neurons. In various embodiments described herein, the machine learning model may be employed in the form of one or more neural network models. For example, the processing units may be arranged in multiple layers (e.g., one or more input layers, one or more hidden layers, or one or more output layers, or a combination thereof) connected by varying connection strengths (e.g., which may be commonly referred to as "weights" in the art). The neural network model may learn through training, where data containing known outcomes is input to the computer model, outputs related to the data are compared to known outcomes, or the weights of the computer model are autonomously adjusted based on the comparison to reproduce the known outcomes, or a combination thereof. As used herein, the term "training data" may refer to data and / or data sets used to train one or more neural network models. As a neural network model trains (e.g., utilizes more training data), the computer model can become increasingly accurate, and thus the trained neural network model can accurately analyze data, including unknown outcomes, based on learning knowledge from the training data to facilitate one or more machine learning tasks. Exemplary neural network models include perceptron (P), feed forward (FF), radial basis network (RBF), deep feed forward (DFF), and eigenvalue (Eigenvalue).forward), recurrent neural network (RNN), long / short term memory (LSTM), gated recurrent unit (GRU), autoencoder (AE), variational AE (VAE), denoising AE (DAE), sparse AE (SAE), markov chain (MC), Hopfield network (HN), Boltzmann machine (BM), deep belief network (DBN), deep convolutional network (DCN), deconvolutional network (DN), deep convolutional inverse graphics network (DCIGN), generative adversarial network (GAN) The neural network may include, but is not limited to, a liquid state machine (LSM), an extreme learning machine (ELM), an echo state network (ESN), a deep residual network (DRN), a Kohonen network (KN), a support vector machine (SVM), or a neural Turing machine (NTM), or any combination thereof.

[0022] FIG. 1 illustrates a block diagram of an exemplary, non-limiting system 100 that can discover dependencies in multivariate time series data. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. Aspects of the systems (e.g., system 100, etc.), devices, or processes in various embodiments of the invention can comprise one or more machine-executable components embodied in one or more machines (e.g., embodied in one or more computer-readable media associated with one or more machines). Such components, when executed by one or more machines (e.g., computers, computing devices, virtual machines, etc.), can cause the machines to perform the described operations.

[0023] As shown in FIG. 1, the system 100 may include one or more servers 102, one or more networks 104, input devices 106, and / or observable systems 108. The server 102 may include a time series analysis component 110. The time series analysis component 110 may further include a communication component 112. The server 102 may also include or be otherwise associated with at least one memory 116. The server 102 may further include a system bus 118 that may couple to various components, such as, but not limited to, the time series analysis component 110 and associated components, the memory 116, and / or a processor 120. Although the server 102 is shown in FIG. 1, in other embodiments, multiple devices of various types may be associated with or include the features shown in FIG. 1. Additionally, the server 102 may be in communication with one or more cloud computing environments.

[0024] The one or more networks 104 may include wired and wireless networks, including, but not limited to, a cellular network, a wide area network (WAN) (e.g., the Internet), or a local area network (LAN). For example, the server 102 may communicate with the one or more input devices 106 and / or the observable system 108 (and vice versa) using substantially any desired wired or wireless technology, including, but not limited to, cellular, WAN, wireless fidelity (Wi-Fi), Wi-Max, WLAN, or Bluetooth technology, or a combination thereof. Additionally, in the illustrated embodiment, the time series analysis component 110 may be provided on the one or more servers 102, although it should be understood that the architecture of the system 100 is not so limited. For example, the time series analysis component 110 or one or more components of the time series analysis component 110 may reside on another computing device, such as another server device, a client device, or the like.

[0025] The one or more input devices 106 may include one or more computerized devices, including, but not limited to, a personal computer, a desktop computer, a laptop computer, a mobile phone (e.g., a smartphone), a computerized tablet (e.g., equipped with a processor), a smart watch, a keyboard, a touch screen, or a mouse, or combinations thereof. The one or more input devices 106 may be employed to input one or more model settings, thresholds, or selections, or combinations thereof, of the various embodiments described herein, into the system 100, thereby sharing such data with the server 102 (e.g., via a direct connection, or via one or more networks 104, or both). For example, the one or more input devices 106 may transmit data to the communication component 112 (e.g., via a direct connection, or via one or more networks 104, or both). Additionally, the one or more input devices 106 may include one or more displays capable of presenting one or more outputs generated by the system 100 to a user. For example, the one or more displays may include, but are not limited to, a cathode ray tube display (CRT), a light-emitting diode display (LED), an electroluminescent display (ELD), a plasma display panel (PDP), a liquid crystal display (LCD), or an organic light-emitting diode display (OLED), or combinations thereof.

[0026] In various embodiments, one or more input devices 106 and / or one or more networks 104 can be employed to input one or more settings and / or commands into the system 100. For example, in various embodiments described herein, one or more input devices 106 can be employed to run and / or operate the server 102 and / or associated components. Additionally, one or more input devices 106 can be employed to display one or more outputs (e.g., displays, data, and / or visualizations) generated by the server 102 and / or associated components. Additionally, in one or more embodiments, one or more input devices 106 can be included within and / or operably coupled to a cloud computing environment.

[0027] In one or more embodiments, the one or more observable systems 108 can be one or more mechanical and / or computer systems that are the subject of observation by the time series analysis component 110. Additionally, the one or more observable systems 108 can include one or more data collection components 122. The one or more data collection components 122 can measure, detect, collect, and / or aggregate multivariate time series data regarding the one or more mechanical and / or computer systems. For example, the multivariate time series data can include one or more parameter values ​​that characterize one or more measurable properties of the one or more mechanical and / or computer systems. In various embodiments, the one or more data collection components 122 can include one or more sensors. Exemplary sensors can include, but are not limited to, pressure sensors, vibration sensors, temperature sensors, microphones, barometers, accelerometers, piezoelectric sensors, and / or motion sensors, and the like. For example, the one or more measurable properties can be detected and / or measured by the one or more sensors. In another example, the one or more measurable characteristics can relate to transactions, management, or modifications of data, or a combination thereof, etc. One or more data collection components 122 can generate, aggregate, collect, and / or share (e.g., via a direct electrical connection or one or more networks 104) time series data with the time series analysis component 110, or a combination thereof.

[0028] The communications component 112 can receive one or more settings and / or selections entered by one or more input devices 106 and share these settings and / or selections with associated components of the time series analysis component 110. Additionally, the communications component 112 can receive multivariate time series data collected by one or more observable systems 108 (e.g., data collection component 122) and share this multivariate time series data with associated components of the time series analysis component 110. In various embodiments, the communications 112 can communicate with one or more input devices 106 and / or observable systems 108 (e.g., data collection component 122) via a direct electrical connection or one or more networks 104 or both.

[0029] In various embodiments, the time series analysis component 110 may use one or more deep learning neural networks employing Bayesian inference to analyze the time series data made available by one or more data collection components 122 to discover dependencies between various time series. Multivariate time series data (MTS) consisting of D variables (e.g., component time series) may be represented herein according to the following Equation 1:

[0030]

number

number

number

number

[0031] 2 illustrates a diagram of an exemplary non-limiting system 100 further comprising an encoder component 202 in accordance with one or more embodiments described herein. Repetition of descriptions of similar elements employed in other embodiments described herein is omitted for brevity. In various embodiments, the encoder component 202 can encode the multivariate time series data by matching different data sampling rates across a set of constructed multivariate time series data, thereby making the data scalable and parallelizable.

[0032] In one or more embodiments, the encoder component 202 can learn intermediate feature vectors of the multivariate time series data at timestamp t by various encoding networks of the deep learning neural network model employed by the time series analysis component 110. For example, the intermediate feature vector may be, for D input time series at time t,

number

number

[0033] The encoder component 202 can build encoding networks of various versions of the deep learning neural network model employed by the time series analysis component 110 based on transformer network techniques that can increase efficiency by removing recurrent connections. For example, the encoder component 202 can employ a multi-head attention structure. For each i-th time series of a component, the encoder component 202 can generate a

number

number

number

number

number

[0034] 3 illustrates a diagram of an exemplary non-limiting system 100 further comprising an aggregation component 302 in accordance with one or more embodiments described herein. Repetition of descriptions of similar elements employed in other embodiments described herein is omitted for brevity. In various embodiments, the aggregation component 302 can aggregate feature encoding vectors generated by the encoding component 202. For example, the aggregation component 302 can aggregate multiple encoding vectors generated by one or more encoding components 202 and / or encoding networks in preparation for subsequent distribution to one or more further layers of a deep learning neural network model employed by the time series analysis component 110.

[0035] FIG. 4 illustrates a diagram of an exemplary non-limiting system 100 further comprising an attention decoder component 402 and / or a prediction component 404 in accordance with one or more embodiments described herein. Repetition of descriptions of similar elements employed in other embodiments described herein is omitted for brevity. In various embodiments, the attention decoder component 402 can use one or more attention network layers of the deep learning neural network model employed by the time series analysis component 110 to regularize attention among the encoded multivariate time series data based on one or more uncertainty measures (e.g., Shannon entropy values). For example, the one or more attention network layers developed by the attention decoder component 402 can penalize attention configurations of the encoded multivariate time series data that result in uncertainty measures (e.g., Shannon entropy values) greater than a defined threshold (e.g., high and unstable Shannon entropy values). Additionally, the attention decoder component 402 can further generate one or more uncertainty quantifications for the one or more discovered dependencies by Monte Carlo sampling performed in the attention network layer of the machine learning model.

[0036] In one or more embodiments, the attention decoder component 402 outputs, for each d-th time series,

number

number

number

number

[0037]

number

number

number

[0038] for example,

number

number

number

number

number

number

number

number

[0039] In one or more embodiments, the attention decoder component 402 can learn a posterior distribution of the parameters controlling the attention network by Bayesian inference. For example, this distribution can represent the uncertainty in attention and can be analyzed by the attention decoder component 402 to estimate a confidence value in discovering time series dependencies. For example, the neural network parameters can be estimated by ω A (e.g., the subscript A can represent attention in the model), (e.g.,

number

number

number

[0040]

number

number

[0041] Having determined the posterior distribution over all of the network parameters, the attention decoder component 402 then calculates the posterior distribution of the attention variables

number

[0042]

number

number

[0043] ω AGiven the large number of continuous variables involved, directly computing the integrals in Equations 3-4 may be intractable by traditional Bayesian theorem. However, the attention decoder component 402 can make such computations possible by utilizing variational inference techniques to approximate them. For example, the attention decoder component 402 can compute the integrals in Equations 3-4 by applying the ground truth posterior distribution Pr(ω A |h t ) we use a family of distributions Q parameterized by θ θ (ω A ) can be chosen as the variational posterior distribution. Q θ (ω A ) can have the form of a known function, and the attention decoder component 402 computes the true distribution Pr(ω A |h t The parameter θ can be calculated such that the Kullback-Leibler (KL) distance to the vector θ can be minimized.

[0044]

number

[0045] The KL distance can be called the relative entropy and can be an effective tool to quantify the distance between two data distributions. The KL distance can be zero if the two distributions are identical. However, this measure is not symmetric, and the attention decoder component 402 applies a Q-value to the machine learning model so that the machine learning model is refined according to the posterior distribution. A (ω A ) can be forced to learn (e.g., Pr(ω A |h t ) is zero, Q θ (ω A ) can be zero).

[0046] Learned variational distribution Q θ (ω A), the attention decoder component 402 can effectively approximate Equation 4 by Monte Carlo sampling. In particular,

number

number

number

[0047] The attention network developed by the attention decoder component 402

number

[0048]

number

[0049]

number

[0050] As the data proves,

number

[0051]

number

[0052] If there is no preselection, Q θ (ω A A common choice for θ can be a normal distribution. Additionally, the attention decoder component 402 can employ stochastic gradient descent by making the loss function differentiable. All expectation terms are then scaled with Q, which includes the parameters being optimized. θ (ω A ), the reparameterized gradients can be applied to the backpropagation. For example, the attention decoder component 402 first

number

[0053] Pr(h t |ω A ) is the optimal Q that sufficiently approximates θ * (ω A ), the prediction component 404 predicts the D time series at timestamp t according to Equation 9:

number

number

[0054]

number

number

number

number

[0055] For inference, in effect, we can calculate (e.g., by the reparameterization techniques described herein) Q θ (ω A ) K times ω A The first sampling of the sampled ω A Every

number

number

[0056] 5 illustrates a diagram of an exemplary, non-limiting learning objective function 500 that may be employed by one or more attention decoder components 402 and / or prediction components 404 in accordance with one or more embodiments described herein. Repetition of descriptions of similar elements employed in other embodiments described herein has been omitted for brevity. FIG. 5 may illustrate the learning objective function 500 in terms of a neural network in which interconnections between multiple processing units 502 are depicted by dashed lines.

[0057] As described herein and shown in FIG. 5, the attention decoder component 402 can employ a multilayer perceptron (MLP) neural network to learn the variational posterior distribution. The mixture model θ (e.g., a Gaussian model) can allow parameters related to the boundaries of the machine learning model to be encoded (e.g., by the encoder component 202). ω A The value of is the variational posterior distribution Q θ (ω A ) or the corresponding hidden vector h 1 ,h 2 ,...h m Notice regarding a 1 ,a 2 ,...a m The prior distribution Pr can be employed by the attention decoder component 402 to decode the Φ (ω A ) can be parameterized and tunable (e.g., including mixture models).

[0058] KL(Q θ (ω A )||Pr Φ (ω A)) can indicate the discrepancy between the variational posterior and prior distributions and can be further added to the overall learning objective function 500 by the attention decoder component 402 according to various embodiments described herein. Furthermore, the H(a) term can be employed by the attention decoder component 402 to calculate the entropy over the attention weights. For example, the attention information a 1 ,a 2 ,...a D The weights of are sampled w and h 1 ,h 2 ,...h D As described in various embodiments herein and illustrated in FIG. 5, the attention decoder component 402 can further add H(a) to the overall loss function, which can act as a regularization term that can penalize the machine learning model and generate uniform attention. As also illustrated in FIG. 5, k h 1 ,h 2 ,...h D The weights of the aggregated context vectors from can be denoted as the weights of the k-th sequence, which can then serve as inputs to the final output, decoded as the next value in the sequence k. Furthermore, the weights of the one-time Monte Carlo sampling from the posterior distribution, ω, can be denoted as the weights of the aggregated context vectors from A Each vector computed from may be output as a probabilistic attention. In various embodiments, the resulting output may be shared with one or more input devices 106 (e.g., via the communications component 112 and / or one or more networks 104).

[0059] 6 illustrates a flow diagram of an exemplary non-limiting method 600 that can facilitate analysis of multivariate time series data by the system 100 in accordance with one or more embodiments described herein. Repetition of descriptions of similar elements employed in other embodiments described herein has been omitted for the sake of brevity.

[0060] At 602, the method 600 may include defining a neural network component based on input multivariate time series (MTS) data (e.g., by the time series analysis component 110) and distributing each time series (TS) component of the data to each component (e.g., by the time series analysis component 110). For example, the MTS data may be input by one or more data collection components 122 according to various embodiments described herein. At 604, the method 600 may include generating an encoding for each of the distributed TS (e.g., by the encoder component 202). For example, step 604 may include encoding one or more intermediate feature vectors for each of the distributed TS according to various embodiments described. At 606, the encoding at 604 may be analyzed to confirm convergence. If the encoding achieves convergence, the method 600 may proceed to 608; if the encoding does not converge, the method 600 may repeat 604 until convergence is achieved.

[0061] At 608, the method 600 may include aggregating the encoded data (e.g., by the aggregation component 302) and then distributing the encoded data to one or more attention decoders. At 610, the method 600 may include implementing one or more attention mechanisms (e.g., by the attention decoder component 402 and / or the prediction component 404) to analyze the encoded vectors. As shown in FIG. 6, the implementation of the attention mechanism may be performed multiple times (e.g., simultaneously or sequentially) for each TS. For example, one or more attention mechanisms may be implemented according to various features described herein, such as Equations 1-9 and / or the learning objective function shown in FIG. 5.

[0062] At 612, the attention values ​​resulting from the attention mechanism at 610 can be evaluated to determine whether these values ​​are associated with a small uncertainty measure (e.g., Shannon entropy value) and a high prediction accuracy. A defined entropy threshold (e.g., set via one or more input devices 106) can be employed to evaluate whether the uncertainty measure (e.g., Shannon entropy value) is sufficiently small to be considered acceptable. Additionally, a defined confidence threshold (e.g., set via one or more input devices) can be employed to evaluate whether the associated confidence value corresponds to an expected high prediction accuracy. If the entropy value is found to be too high (e.g., greater than a defined entropy threshold) and / or the prediction accuracy is found to be too low (e.g., lower than a defined confidence threshold), method 600 can repeat step 610. If the entropy value is found to be sufficiently small (e.g., below a defined entropy threshold) and / or the prediction accuracy is found to be sufficiently high (e.g., above a defined confidence threshold), method 600 may proceed to step 614. At 614, method 600 may include reporting (e.g., by the time series analysis component 110, the communication component 112, the network 104, and / or the input device 106) the interdependencies of each TS with respect to other TSs of the MTS, along with certainty and / or prediction values.

[0063] 7-8 show exemplary non-limiting graph and chart illustrations that can illustrate the effectiveness of the system 100 according to one or more embodiments described herein. Repetition of descriptions of similar elements employed in other embodiments described herein is omitted for brevity. For example, FIG. 7 shows forecast values, uncertainty values, and / or dependency values ​​that can be generated by the time series analysis component 110. The time series shown in FIG. 7 can be for temperature data measured by one or more temperature sensors of one or more data collection components 122. As shown in FIG. 7, graph 701 shows a time series forecast mapped against later measured time series values, graph 704 shows uncertainty values ​​associated with the forecasts and / or dependencies determined by the time series analysis component 110, and graph 706 shows dependencies within the multivariate time series data discovered by the time series analysis component 110. With respect to graph 701, line 702 can represent predicted MTS data and line 703 can represent measured MTS data, with lines 702 and 703 overlapping considerably (e.g., thereby indicating high accuracy of prediction). As shown in graph 706, one or more observable systems 108 can be characterized by multiple measurable characteristics, including temperature, actual pressure, a first silicon controlled rectifier (SCR) voltage (e.g., “internal SCR voltage”), a second SCR voltage (e.g., “external SCR voltage”), or SCR power (e.g., “SCR power SP”), or combinations thereof. The multiple measurable characteristics can be ascertained by one or more sensors of data collection component 122 and characterized by multivariate time series data. Furthermore, dependencies between the measurable characteristics can exist within the observed system, as those dependencies can be discovered by time series analysis component 110, as reported in graph 706.

[0064] Chart 800 and graph 802 are for synthetic data used to demonstrate the effectiveness of system 100. Synthetic multivariate time series data contains known ground truth of dependencies and can be used to validate dependency predictions generated by time series analysis component 110. The results shown in chart 800 and graph 802 are for analysis of 10 time series components by time series analysis component 110. As shown in FIG. 8, entropy regularization incorporated into the attention mechanism employed by attention decoder component 402 can increase the accuracy of dependencies discovered and / or predictions generated.

[0065] 9 illustrates a flow diagram of an exemplary non-limiting computer-implemented method 900 that can facilitate time series data analysis by the system 100 in accordance with one or more embodiments described herein. Repetitive descriptions of similar elements employed in other embodiments described herein have been omitted for the sake of brevity.

[0066] At 902, the computer-implemented method 900 may include receiving, by the system 100 operatively coupled to the processor 120 (e.g., via the communications component 112), multivariate time series data regarding one or more observable systems 108. For example, the multivariate time series data may include one or more measurable characteristics that may characterize the one or more observable systems 108. For example, according to various embodiments described herein, the multivariate time series data may be collected, measured, detected, or generated by one or more data collection components 122, or a combination thereof.

[0067] At 904, the computer-implemented method 900 may include generating (e.g., by the time series analysis component 110) one or more machine learning models (e.g., neural network models) capable of discovering one or more dependencies between the multivariate time series data based on one or more uncertainty measures (e.g., Shannon entropy values) and using one or more attention mechanisms. For example, the one or more attention mechanisms may be implemented according to various embodiments described herein, including the learning objective functions shown in Equations 1-9 and / or FIG.

[0068] 10 illustrates a flow diagram of an exemplary non-limiting computer-implemented method 1000 that can facilitate time series data analysis by the system 100 in accordance with one or more embodiments described herein. Repetitive descriptions of similar elements employed in other embodiments described herein have been omitted for the sake of brevity.

[0069] At 1002, the computer-implemented method 1000 may include receiving, by the system 100 operatively coupled to the processor 120 (e.g., via the communication component 112), multivariate time series data regarding one or more observable systems 108. For example, the multivariate time series data may include one or more measurable characteristics that may characterize the one or more observable systems 108. For example, according to various embodiments described herein, the multivariate time series data may be collected, measured, detected, or generated by one or more data collection components 122, or a combination thereof.

[0070] At 1004, the computer-implemented method 1000 may include, by the system 100, encoding (e.g., by the encoder component 202) the multivariate time series data by matching different data sampling rates across the constructed set of multivariate time series data, where the encoding at 1004 may make the multivariate time series data scalable and parallelizable. At 1006, the computer-implemented method 1000 may include, by the system 100, regularizing attention (e.g., by the attention decoder component 402) between the encoded multivariate time series data based on an uncertainty measure (e.g., a Shannon entropy value) using one or more attention network layers of a machine learning model (e.g., a neural network model) to discover one or more dependencies among the multivariate time series data.

[0071] At 1008, the computer-implemented method 1000 can include penalizing (e.g., by the attention decoder component 402) attention configurations of the encoded multivariate time series data that result in an uncertainty measure (e.g., Shannon entropy value) greater than a defined threshold (e.g., thereby penalizing results with high entropy values ​​that can be associated with instability and low prediction accuracy), by the system 100. At 1010, the computer-implemented method 1000 can include generating (e.g., by the attention decoder component 402) one or more uncertainty quantifications for the dependencies via Monte Carlo sampling performed at the attention network layer.

[0072] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the subject matter presented herein is not limited to a cloud computing environment. Embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0073] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computational resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) and for rapidly provisioning and releasing these resources with minimal administrative effort or interaction with the service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0074] The features are as follows:

[0075] On-Demand Self-Service: Cloud customers can automatically provision server time and computing power, such as network storage, as needed, without the need for unilateral human interaction with the service provider.

[0076] Broad network access: Cloud capabilities are available over the network and can be accessed using standard mechanisms, facilitating usage by heterogeneous thin- or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0077] Resource Pool: The provider's computing resources are pooled and offered to multiple consumers using a multi-tenant model. Different physical and virtual resources are dynamically allocated and reallocated according to demand. There is a sense of location independence, consumers usually have no control or knowledge regarding the exact location of the resources offered, although at a higher level of abstraction they may be able to specify a location (e.g. country, state, or data center).

[0078] Rapid Elasticity: Cloud capacity can be provisioned quickly and elastically, in some cases automatically, to scale out quickly, and quickly released to scale in quickly. Capacity available for provisioning often appears to consumers as unlimited, available for purchase in any amount at any time.

[0079] Metered Services: Cloud systems leverage metering to automatically control and optimize resource usage at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services utilized.

[0080] The service model is as follows:

[0081] SaaS (Software as a Service): The capability provided to the consumer is the use of the provider's applications running on a cloud infrastructure. Those applications can be accessed from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or individual application functionality, except for the possibility of limited user-specific application configuration settings.

[0082] PaaS (Platform as a Service): The capability provided to a consumer is to deploy applications that the consumer creates or acquires, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0083] Infrastructure as a Service (IaaS): The capability provided to a consumer is the provisioning of processing, storage, network, and other basic computing resources over which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, and deployed applications, and in some cases has limited control over selected network components (e.g., host firewalls).

[0084] The deployment model is as follows:

[0085] Private Cloud: This cloud infrastructure is operated solely for the organization. It can be managed by the organization or a third party and can exist on-premise or off-premise.

[0086] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It can be managed by these organizations or a third party and can reside on-premise or off-premise.

[0087] Public Cloud: This cloud infrastructure is available for use by general users or large industry organizations and is owned by an organization that sells cloud services.

[0088] Hybrid cloud: This cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain unique but are bound together by standardized or proprietary technologies that allow for the portability of data and applications (e.g., cloud bursting to balance the load between clouds).

[0089] A cloud computing environment is a service-oriented environment with an emphasis on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0090] 11, an exemplary cloud computing environment 1100 is illustrated. As illustrated, the cloud computing environment 1100 includes one or more cloud computing nodes 1102 with which local computing devices used by cloud consumers (e.g., a personal digital assistant (PDA) or mobile phone 1104, a desktop computer 1106, a laptop computer 1108, and / or an automobile computer system 1110, etc.) can communicate. The nodes 1102 may communicate with each other. The nodes 1102 may be physically or virtually grouped together in one or more networks (not shown), such as in a private cloud, community cloud, public cloud, or hybrid cloud, or combination thereof, as previously described herein. This allows the cloud computing environment 1100 to provide an infrastructure, platform, and / or SaaS that does not require the cloud consumer to maintain resources on a local computing device. The types of computing devices 1104-1110 shown in FIG. 11 are intended to be illustrative only, and it will be understood that the computing nodes 1102 and cloud computing environment 1100 can communicate with any type of computer-controlled device via any type of network and / or network-addressable connection (e.g., a connection using a web browser).

[0091] Referring now to Figure 12, a set of functional abstraction layers provided by cloud computing environment 1100 (Figure 11) is shown. Repeated descriptions of similar elements used in other embodiments described herein are omitted for the sake of brevity. It should be understood in advance that the components, layers, and functions shown in Figure 12 are intended to be illustrative only, and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0092] The hardware and software layer 1202 includes hardware and software components. Examples of hardware components include mainframe 1204, reduced instruction set computer (RISC) architecture based server 1206, server 1208, blade server 1210, storage device 1212, and network and network components 1214. In some embodiments, the software components include network application server software 1213 and database software 1218.

[0093] The virtualization layer 1220 comprises an abstraction layer that can provide virtual entities such as virtual servers 1222 , virtual storage 1224 , virtual networks including virtual private networks 1226 , virtual applications and operating systems 1228 , and virtual clients 1230 .

[0094] As an example, the management layer 1232 may provide the following functions: Resource provisioning 1234 dynamically procures computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 1236 tracks costs as resources are utilized within the cloud computing environment and generates and sends bills for the utilization of those resources. As an example, those resources may include application software licenses. Security verifies the identity of cloud users and tasks, and protects data and other resources. User portal 1238 provides users and system administrators with access to the cloud computing environment. Service level management 1240 allocates and manages cloud computing resources to meet required service levels. Service level agreement (SLA) planning and execution 1242 pre-provisions and procures cloud computing resources in anticipation of upcoming demands according to SLAs.

[0095] Workload layer 1244 illustrates examples of functionality available in a cloud computing environment. Examples of workloads and functionality provided from this layer include mapping and navigation 1246, software development and lifecycle management 1248, virtual classroom instruction delivery 1250, data analytics processing 1252, transaction processing 1254, and multivariate time series analysis 1256. Various embodiments of the present invention may utilize the cloud computing environments described with reference to Figures 11 and 12 to collect multivariate time series data and / or employ one or more attention mechanisms described herein (e.g., via machine learning models).

[0096] The present invention may be a system, a method, or a computer program product, or a combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer readable storage medium that includes computer readable program instructions for causing a processor to perform aspects of the present invention. The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes portable floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves in which the instructions are recorded, and any suitable combination of these. As used herein, a computer-readable storage medium should not itself be construed as a transitory signal, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted over a wire.

[0097] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing device / processing device or to an external computer or storage device over a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof) that may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing device / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions for storage on a computer readable storage medium within each computing device / processing device.

[0098] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, to carry out aspects of the invention, electronic circuitry including, for example, programmable logic circuitry, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer readable program instructions to customize the electronic circuitry by utilizing state information of the computer readable program instructions.

[0099] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, are implemented by computer readable program instructions.

[0100] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, produce means for performing the functions / operations specified in the blocks of the flowcharts and / or block diagrams. These computer readable program instructions may be stored on a computer readable storage medium in a specific manner such that the computer readable storage medium on which the instructions are stored comprises an article of manufacture including instructions for performing aspects of the functions / operations specified in the blocks of the flowcharts and / or block diagrams, and may direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function.

[0101] Computer readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / acts specified in the blocks of the flowcharts and / or block diagrams, thereby causing a series of operable steps to be performed on the computer, other programmable apparatus, or other device to generate a computer-implemented process.

[0102] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously or in reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flowchart illustrations, as well as combinations of blocks included in the block diagrams and / or flowchart illustrations, are implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0103] To provide further context for the various embodiments described herein, Figure 13 and the following discussion are intended to provide a general overview of a suitable computing environment 1300 in which various embodiments described herein may be implemented. Although the embodiments are described above in the general context of computer-executable instructions that may be executed on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules, or as a combination of hardware and software, or both.

[0104] Typically, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Additionally, those skilled in the art will appreciate that the methods of the present invention can be practiced using other computer system configurations, including single or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which is operatively coupled to one or more associated devices.

[0105] The embodiments described herein may also be practiced in a distributed computing environment where certain tasks are performed by remote processing devices linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices. For example, in one or more embodiments, computer-executable components may be executed from a memory that may include or consist of one or more distributed memory units. As used herein, the terms "memory" and "memory unit" are interchangeable. Furthermore, one or more embodiments described herein may execute code of computer-executable components in a distributed manner, e.g., with multiple processors coupled or working in concert to execute code from one or more distributed memory units. As used herein, the term "memory" may encompass a single memory or memory unit at one location or multiple memories or memory units at one or more locations.

[0106] A computing device typically includes various media, which may include computer-readable storage media, machine-readable storage media, or communication media, or a combination thereof, and in this specification, the two terms are used differently from each other as follows. A computer-readable storage medium or machine-readable storage medium can be any available storage medium that can be accessed by a computer, and includes both volatile and non-volatile media, removable and non-removable media. By way of example, a computer-readable storage medium or machine-readable storage medium can be implemented in association with any method or technology for storage of information, such as, but not limited to, computer-readable or machine-readable instructions, program modules, structured or unstructured data, etc.

[0107] A computer-readable storage medium may include, but is not limited to, random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD), Blu-ray disc (BD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, solid state drive or other solid state storage device, or other tangible and / or non-transitory medium used to store the desired information. In this regard, the terms "tangible" or "non-transitory" as applied to storage, memory, or computer-readable medium should be understood to exclude only propagating, ephemeral signals as modifiers and not to waive all rights to all standard storage, memory, or computer-readable media that are not themselves merely propagating, ephemeral signals.

[0108] The computer-readable storage medium can be accessed by one or more local or remote computing devices for various operations on the information stored by the medium, such as via access requests, queries, or other data retrieval protocols.

[0109] Communication media typically embodies computer readable instructions, data structures, program modules, or other structured or unstructured data in a data signal, such as a modulated data signal (e.g., carrier wave or other transport mechanism) and includes any information delivery or transport media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal or signals. By way of example, and not limitation, communication media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared and other wireless media.

[0110] Referring again to FIG. 13, an exemplary environment 1300 for implementing various embodiments of the aspects described herein includes a computer 1302 including a processing unit 1304, a system memory 1306, and a system bus 1308. The system bus 1308 couples system components including, but not limited to, the system memory 1306 to the processing unit 1304. The processing unit 1304 can be any of a variety of commercially available processors. Dual microprocessors and other multi-processor architectures can also be employed as the processing unit 1304.

[0111] The system bus 1308 can be any of several types of bus structures that can be further interconnected into a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 1306 includes a ROM 1310 and a RAM 1312. A basic input / output system (BIOS) can be stored in a non-volatile memory, such as a ROM, erasable programmable read only memory (EPROM), or EEPROM, and the BIOS contains basic routines that help transfer information between elements within the computer 1302, such as during startup. The RAM 1312 can also include a high-speed RAM, such as a static RAM, for caching data.

[0112] The computer 1302 further includes an internal hard disk drive (HDD) 1314 (e.g., EIDE, SATA), one or more external storage devices 1316 (e.g., a magnetic floppy disk drive (FDD), a memory stick or flash drive reader, a memory card reader, etc.), and an optical disk drive 1320 (e.g., a drive that can read from or write to CD-ROM disks, DVDs, BDs, etc.). Although the internal HDD 1314 is shown as being within the computer 1302, the internal HDD 1314 can also be configured for use within an external suitable enclosure (not shown). Additionally, although not shown within the environment 1300, a solid state drive (SSD) can be used in addition to or in place of the HDD 1314. The HDD 1314, external storage device 1316, and optical disk drive 1320 can be connected to the system bus 1308 by a HDD interface 1324, an external storage interface 1326, and an optical drive interface 1328, respectively. The interface 1324 for external drive implementations can include at least one or both of Universal Serial Bus (USB) interface technology and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technology. Other external drive connection technologies are within the contemplation of the embodiments described herein.

[0113] The drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. The drives and storage media of the computer 1302 accommodate the storage of any data in a suitable digital format. Although the above description of computer-readable storage media refers to various storage devices, it should be understood by those skilled in the art that other types of storage media that can be read by a computer, whether currently existing or developed in the future, can also be used within the exemplary operating environment, and further, any such storage media can include computer-executable instructions for performing the methods described herein.

[0114] A number of program modules, including an operating system 1330, one or more application programs 1332, other program modules 1334, and program data 1336, may be stored in the drives and RAM 1312. All or portions of the operating system, applications, modules, and / or data may also be cached in RAM 1312. The systems and methods described herein may be implemented using various commercially available operating systems or combinations of operating systems.

[0115] The computer 1302 may optionally include emulation technology. For example, a hypervisor (not shown) or other intermediary may emulate the hardware environment of the operating system 1330, and the emulated hardware may optionally be different than the hardware shown in FIG. 13. In such an embodiment, the operating system 1330 may include one of a number of virtual machines (VMs) hosted on the computer 1302. Additionally, the operating system 1330 may provide a runtime environment, such as the Java® runtime environment or the .NET framework, to the application 1332. The runtime environment is a consistent execution environment that allows the application 1332 to run on any operating system that includes the runtime environment. Similarly, the operating system 1330 may support containers, and the application 1332 may be in the form of a container, which is a lightweight, standalone, executable package of software that includes, for example, code, runtime, system tools, system libraries, and settings for the application.

[0116] Additionally, computer 1302 can be enabled with a security module, such as a trusted processing module (TPM). For example, with a TPM, a boot component can hash the next boot component in time and wait for a match of the hash result with a protected value before loading the next boot component. This process can occur at any layer in the code execution stack of computer 1302, for example, applied at the application execution level or at the operating system (OS) kernel level, thereby enabling security at any level of code execution.

[0117] A user may enter commands and information into the computer 1302 via one or more wired / wireless input devices (e.g., a keyboard 1338, a touch screen 1340, and a pointing device such as a mouse 1342). Other input devices (not shown) may include a microphone, an infrared (IR) remote control, a radio frequency (RF) remote control, or other remote control, a joystick, a virtual reality controller and / or a virtual reality headset, a game pad, a touch pen, an image input device (e.g., a camera), a gesture sensor input device, a vision movement sensor input device, an emotion or face detection device, a biometric input device (e.g., a fingerprint or iris scanner), and the like. These and other input devices are often connected to the processing unit 1304 via an input device interface 1344 that may be coupled to the system bus 1308, but may be connected by other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, a BLUETOOTH® interface, and the like.

[0118] A monitor 1346 or other type of display device can also be connected to the system bus 1308 via an interface, such as a video adapter 1348. In addition to the monitor 1346, a computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc.

[0119] The computer 1302 can operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 1350, via wired and / or wireless communications. The remote computer 1350 can be a workstation, a server computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and typically includes many or all of the elements described relative to the computer 1302, although for purposes of brevity, only a memory / storage device 1352 is shown. The logical connections shown in the figure include wired / wireless connections to a local area network (LAN) 1354 and / or larger networks (e.g., a wide area network (WAN) 1356). Such LAN and WAN networking environments are commonplace in offices and companies, facilitating enterprise-wide computer networks, such as intranets, all of which may be connected to a global communications network (e.g., the Internet).

[0120] When used in a LAN networking environment, the computer 1302 can be connected to the local network 1354 via a wired and / or wireless communication network interface or adapter 1358. The adapter 1358 can facilitate wired or wireless communication with the LAN 1354, and the LAN 1354 can also include a wireless access point (AP) disposed for communicating with the adapter 1358 in a wireless mode.

[0121] When used in a WAN network environment, the computer 1302 can include a modem 1360 or can be connected to a communications server on the WAN 1356 by other means for establishing communications over the WAN 1356, such as via the Internet. The modem 1360, which can be a wired or wireless device present internally or externally, can be connected to the system bus 1308 via the input device interface 1344. In a networked environment, program modules depicted relative to the computer 1302, or portions thereof, can be stored in the remote memory / storage device 1352. It will be understood that the network connections shown are examples and other means of establishing a communications link between the computers can be used.

[0122] When used in either a LAN or WAN network environment, the computer 1302 can access a cloud storage system or other network-based storage system in addition to or instead of the external storage device 1316 as previously described. In general, a connection between the computer 1302 and the cloud storage system can be established via the LAN 1354 or the WAN 1356, for example, by an adapter 1358 or a modem 1360, respectively. Upon connecting the computer 1302 to an associated cloud storage system, the external storage interface 1326 can manage the storage provided by the cloud storage system, as well as other types of external storage, using the adapter 1358 or the modem 1360, or both. For example, the external storage interface 1326 can be configured to provide access to the cloud storage sources as if the cloud storage sources were physically connected to the computer 1302.

[0123] The computer 1302 may be operable to communicate with any wireless device or entity operatively arranged in wireless communication (e.g., a printer, a scanner, a desktop computer or portable computer or both, a portable data assistant, a communications satellite, any piece or location of equipment associated with a wirelessly detectable tag (e.g., a kiosk, a newsstand, a store shelf, etc.), and a telephone). This communication may include Wireless Fidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, this communication may be of a predefined structure, similar to a conventional network, or may simply be an ad-hoc communication between at least two devices.

[0124] The foregoing includes merely examples of systems, computer program products, and computer-implemented methods. Of course, for purposes of describing this disclosure, it is not possible to describe every conceivable combination of components, products, or computer-implemented methods, or combinations thereof, but one of ordinary skill in the art will recognize that many other combinations and permutations of the present disclosure are possible. Furthermore, to the extent that terms such as "including," "having," "owning," and the like are used in the detailed description, claims, appendices, and drawings, such terms are intended to be inclusive in a manner similar to the term "comprising," as "comprising" is interpreted when used as a transitional term in a claim. The description of the various embodiments is presented for illustrative purposes, but is not intended to be exhaustive, and is not limited to the disclosed embodiments. It will be apparent to one of ordinary skill in the art that many modifications and variations are possible without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications, or technical improvements beyond those found in the marketplace, or to enable other skilled in the art to understand the embodiments disclosed herein.

Claims

1. a memory for storing computer executable components; a processor operatively coupled to the memory for executing the computer-executable components stored in the memory, the computer-executable components comprising: A system that includes a time series analysis component that generates a machine learning model that uses an attention mechanism to discover dependencies between multivariate time series data based on an uncertainty measure.

2. 2. The system of claim 1, further comprising an encoder component that encodes the multivariate time series data by matching different sampling rates of data across a constructed set of the multivariate time series data, wherein encoding renders the multivariate time series data scalable and parallelizable.

3. 3. The system of claim 2, further comprising an attention decoder component that regularizes attention among the encoded multivariate time series data based on the uncertainty measure using an attention network layer of the machine learning model.

4. 4. The system of claim 3, wherein the attention network layer penalizes attention configurations of the encoded multivariate time series data that result in the uncertainty measure being smaller than a defined threshold.

5. The system of claim 3 , wherein the attention decoder component further determines a posterior distribution of parameters controlling the attention network layers by Bayesian estimation.

6. 4. The system of claim 3, wherein the attention decoder component further generates uncertainty quantifications for the dependencies via Monte Carlo sampling performed on the attention network layer.

7. The multivariate time series data characterizes a measurable characteristic, and the system: The system of claim 6 , further comprising a prediction component that predicts a value of the measurable characteristic based on the discovered dependencies and the uncertainty quantification.

8. A computer-implemented method comprising: generating, by a system operatively coupled to a processor, a machine learning model that uses an attention mechanism to discover dependencies among multivariate time series data based on an uncertainty measure.

9. 10. The computer-implemented method of claim 8, further comprising encoding the multivariate time series data by matching different data sampling rates across a constructed set of the multivariate time series data, wherein encoding renders the multivariate time series data scalable and parallelizable.

10. 10. The computer-implemented method of claim 9, further comprising: regularizing, by the system, attention among the encoded multivariate time series data based on the uncertainty measure using an attention network layer of the machine learning model.

11. 11. The computer-implemented method of claim 10, further comprising: penalizing, by the system, attentional configurations of the encoded multivariate time series data that result in the uncertainty measure being less than a defined threshold.

12. The computer-implemented method of claim 10 , further comprising determining, by the system, a posterior distribution of parameters controlling the attention network layer by Bayesian estimation.

13. 11. The computer-implemented method of claim 10, further comprising generating, by the system, uncertainty quantifications for the dependencies via Monte Carlo sampling performed at the attention network layer.

14. The multivariate time series data characterizes a measurable property, and the computer-implemented method further comprises: The computer-implemented method of claim 13 , further comprising predicting, by the system, a value of the measurable characteristic based on the discovered dependencies and the uncertainty quantification.

15. A computer program causing a computer to execute the method according to any one of claims 8 to 14.

16. 16. A computer readable storage medium having the computer program of claim 15 stored thereon.

Images