Ring buffer for generating machine learning estimates for streaming observations in real time

Abstract

Systems, methods, and other embodiments associated with generating ML estimated streams in real-time from observed streams using a ring dual buffer are described. In an example method, observations are received from a stream of observations. The observations are loaded into the ring buffer in real time. The annular buffer includes a first buffer and a second buffer configured together in an annular configuration. An estimate of what the observation is expected is generated from the observation in the ring buffer by a machine learning model. The generation of the estimates alternate between generating estimates from observations in the first buffer in parallel with loading the second buffer and generating estimates from observations in the second buffer in parallel with loading the first buffer. The estimate is written to the estimated stream in real-time after generation.

Description

Background Art

[0001] Sensors for a wide range of physical phenomena can be affixed to machines, equipment, systems, or facilities (collectively, “assets”). Sensors generate data about physical phenomena occurring in or around assets. The data generated by sensors can be monitored or analyzed by computers. Summary of the Invention

[0002] In one embodiment, one or more non-transitory computer-readable media are provided. The non-transitory computer-readable media includes computer-executable instructions stored thereon for generating an estimated flow in real time from an observed flow. When executed by at least a processor of a computer, the computer-executable instructions cause the computer to perform the operations or steps of a method for generating an estimated flow in real time from an observed flow. The instructions cause the computer to receive observations from the observed flow. The instructions cause the computer to load the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. The instructions cause the computer to generate an estimate of what the observations are expected to be from the observations in the circular buffer through a machine learning model. The generation of the estimate alternates between: (1) generating the estimate in parallel from the observations in the first buffer while the second buffer is being loaded, and (2) generating the estimate in parallel based on the observations in the second buffer while the first buffer is being loaded. And, the instructions cause the computer to write the estimate to the estimated flow in real time as the estimate is generated.

[0003] In one embodiment, a computer-implemented method for generating an estimated stream in real time from an observed stream is presented. The method includes receiving observations from the observed stream. The method includes loading the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. The method includes generating, from the observations in the circular buffer, an estimate of what the observations are expected to be by a machine learning model. The generation of the estimate alternates between: (1) generating the estimate from the observations in the first buffer in parallel while the second buffer is being loaded; and (2) generating the estimate from the observations in the second buffer in parallel while the first buffer is being loaded. And, the method includes writing the estimate to the estimated stream in real time as the estimate is generated.

[0004] In one embodiment, a computing system is provided. The computing system includes at least one processor connected to at least one memory and one or more non-transitory computer-readable media. The non-transitory computer-readable media includes computer-executable instructions stored thereon for generating an estimated flow from an observed flow in real time. When executed by the processor having access to at least the memory, the computer-executable instructions cause the computing system to perform the operations or steps of a method for generating an estimated flow from an observed flow in real time. The instructions cause the computing system to receive observations from the observed flow. The instructions cause the computing system to load the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. The instructions cause the computing system to generate an estimate of what the observation is expected to be from the observations in the circular buffer using a machine learning model. The generation of the estimate alternates between: (1) generating the estimate from the observations in the first buffer in parallel while the second buffer is being loaded, and (2) generating the estimate from the observations in the second buffer in parallel while the first buffer is being loaded. And, the instructions cause the computing system to write the estimate to the estimated flow in real time as the estimate is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The accompanying drawings that are incorporated into the specification and form a part thereof have illustrated various systems, methods and other embodiments of the present disclosure.It will be appreciated that the element boundaries (e.g., frame, group of frames or other shapes) illustrated in the figures represent an embodiment of boundaries.In certain embodiments, an element can be implemented as multiple elements, or multiple elements can be implemented as an element.In certain embodiments, the element of the internal assembly of another element can be implemented as an external assembly, and vice versa.In addition, elements can not be drawn to scale.

[0006] Figure 1 An embodiment of a streaming ML estimation system associated with generating a stream of ML estimates in real time from a stream of observations using a circular double buffer is illustrated.

[0007] Figure 2 One embodiment of a streaming ML estimation method associated with generating a stream of ML estimates in real time from an observed stream using a circular double buffer is illustrated.

[0008] Figure 3 Illustrated are range charts for single-buffer operation and for circular double-buffer operation using alternating pointers to switch between buffers.

[0009] Figure 4An example graph showing total latency versus number of signals for streaming ML anomaly detection using a single-buffer processing configuration.

[0010] Figure 5 An example graph showing total latency versus number of signals for streaming ML anomaly detection using a circular double-buffer processing configuration.

[0011] Figure 6 An embodiment of a computing system configured with the disclosed example systems and / or methods is illustrated. DETAILED DESCRIPTION

[0012] This document describes systems, methods, and other embodiments for generating a stream of machine learning (ML) estimates from a stream of observations in real time using a circular double buffer. In one embodiment, a streaming ML estimation system uses a circular double buffer to parallelize input-output activity with ML estimation activity to reduce end-to-end latency. For example, the ML estimation system stores and retrieves observations from two opposing buffers connected together in a circular fashion, such that input / output latency coincides with (and is hidden by) ML processing latency.

[0013] In one embodiment, a streaming ML estimation system receives a stream of incoming observations. Observations in the stream are placed into a circular buffer in real time as they are received. Estimates are generated from the observations in the circular buffer by an ML model. The ML model generates an estimate of what is expected from the observation. While observations are being placed into a first buffer of the circular buffer, the ML model is generating estimates from a second buffer of the circular buffer, and while observations are being placed into the second buffer of the circular buffer, the ML model is generating estimates from the first buffer of the circular buffer. Estimates are written to an output stream in real time as they are generated. The stream of estimates can be used together with the stream of observations to detect anomalies in the stream of observations.

[0014] Any action or function described or claimed herein is not performed by the human mind. Any interpretation that any action or function can be performed by the human mind is inconsistent and contrary to this disclosure.

[0015] -definition-

[0016] As used herein, the terms "stream" and "streaming" refer to the transmission, consumption, and reception of data in a steady, continuous flow (eg, at a sampling rate), allowing for consumption of portions of the data to occur as it is received.

[0017] As used herein, the term "real-time" refers to substantially instantaneous operations that keep up with the throughput of the data stream. In other words, the real-time process occurs within the maximum delay that does not result in a decrease in the throughput of the data stream.

[0018] As used herein, the term "latency" refers to the delay or amount of time it takes to process and / or transmit data. As used herein, the term "end-to-end latency" refers to the delay from the start of a process to the end of the process. For example, end-to-end latency can be the time from when an observation arrives at a streaming ML estimation system to when an estimate for that observation is written to the output stream (as discussed below).

[0019] As used herein, the term "time series" refers to a data structure in which a series of data points (such as observations or sampled values) are indexed in time order. In one embodiment, the data points of a time series can be indexed by an index such as a timestamp and / or an observation number. As used herein, the terms "time series signal" and "time series" are synonymous.

[0020] As used herein, the term "vector" refers to a data structure that includes a set of data points (such as observations or sampled values) at one particular index (such as a timestamp and / or observation number) from multiple time series.

[0021] As used herein, the term "time series database" refers to a data structure that includes one or more time series that share a common index (such as a series of timestamps, positions, or observation numbers). As an example, a time series can be considered a "column" of the time series database, while a vector can be considered a "row" of the time series database.

[0022] As used herein, the term "residual" refers to the difference between a value (such as a measured, observed, sampled, or resampled value) and an estimate, reference, or prediction of what the value is expected to be. In one embodiment, the residual is a positive or negative value. In another embodiment, the residual is an absolute value or a magnitude. In one embodiment, a residual time series or a time series of residuals refers to a time series consisting of the residual values between the time series of values and the time series of what the values are expected to be. And, in one embodiment, a residual vector or a vector of residuals refers to a vector consisting of the residual values between the vector of values and the vector of what the values are expected to be.

[0023] — Example Streaming ML Estimation System —

[0024] Figure 1One embodiment of a streaming ML estimation system 100 is illustrated, which is associated with generating a stream of ML estimates from an observed stream in real time using a circular double buffer. The streaming ML estimation system 100 includes components for generating an estimated stream 105 from an observed stream 110 in real time. The components of the streaming ML estimation system 100 include an input handler 115, a circular double buffer 120, a machine learning model 125, and an output handler 130. In one embodiment, each of these components 115, 120, 125, and 130 of the streaming ML estimation system 100 can be implemented as software executed by computer hardware. For example, the components 110, 115, 120, 125, and 130 can be implemented as one or more intercommunicating software modules, routines, or services for performing the functionality of the components.

[0025] Input handler 115 is configured to receive as input a stream of observations 110 and load individual observations 135 received from the stream of observations 110 into a circular double buffer 120. Circular double buffer 120 is configured to store observations 135 from the input handler and provide observations 140 to machine learning model 125. Machine learning model 125 is configured to generate estimates 145 of what the observations 140 from the circular buffer are expected to be. Output handler 130 is configured to write estimates 145 as output to stream of estimates 105 in real time after generating them.

[0026] The circular double buffer 120 includes a first buffer 150 and a second buffer 155 configured together in a circular configuration. The circular configuration of the first buffer 150 and the second buffer 155 connects the two buffers end to end in a loop. When one of the buffers 150, 155 becomes full and a subsequent write operation is performed, the write is performed on the other buffer 150, 155, overwriting the oldest data. The generation of estimates 145 by the machine learning model 125 alternates between: generating estimates 145 from observations 140 in the first buffer 150 in parallel (as indicated at reference numeral 160) while the second buffer 155 is being loaded (as indicated at reference numeral 165) or contemporaneously therewith, and generating estimates 145 from observations 140 in the second buffer 155 in parallel (as indicated at reference numeral 170) while the first buffer 150 is being loaded (as indicated at reference numeral 175) or contemporaneously therewith.

[0027] In one embodiment, the machine learning model 125 is configured to receive training data 180, a set of observations representing normal, nominal, or typical measured values for the observations. The machine learning model 125 is configured to perform an initial training operation to adjust the machine learning model 125 to produce estimates that are consistent with the training data 180.

[0028] In one embodiment, the machine learning model 125 is executed by a primary computing instance, and the input processing program 115 and the output processing program 130 are executed by a secondary computing instance. The ring buffer 120 (including the first buffer 150 and the second buffer 155) is in a memory or storage location that is accessible to both the primary computing instance and the secondary computing instance.

[0029] In one embodiment, the streaming ML estimation system 100 implements a circular double buffer architecture for parallelized ML estimation and input / output handling. Further details about the streaming ML estimation system 100 are presented below. In one embodiment, reference is made to Figure 2 The operation of the streaming ML estimation system 100 is described with reference to the example streaming ML estimation method 200 shown in FIG. Figure 3 The scope diagrams 300, 305 of the delay decomposition shown in FIG and the reference Figure 4 and Figure 5 Graphs 400 and 500 are shown in FIG. 1 to describe the latency incurred by the operation of the streaming ML estimation system 100 .

[0030] — Example Streaming ML Estimation Method —

[0031] Figure 2 An embodiment of a streaming ML estimation method 200 is illustrated, associated with generating a stream of ML estimates in real time from a stream of observations using a circular double buffer. In one embodiment, the streaming ML estimation method 200 receives observations from a stream of observations. The streaming ML estimation method 200 loads the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. The streaming ML estimation method 200 generates an estimate of what the observation is expected to be from the observations in the circular buffer using a machine learning model. The generation of the estimate alternates between generating the estimate from the observations in the first buffer in parallel while the second buffer is being loaded, and generating the estimate from the observations in the second buffer in parallel while the first buffer is being loaded. The streaming ML estimation method 200 writes the estimate to the stream of estimates in real time as the estimate is generated.

[0032] In one embodiment, the streaming ML estimation method 200 is initiated at a "start" block 205 in response to the streaming ML estimation system determining one or more of the following: (i) an incoming stream of observations has been detected; (ii) an instruction has been received to execute the streaming ML estimation method 200 on the stream of observations; (iii) a user or administrator of the streaming ML estimation system 100 has initiated the streaming ML estimation method 200; (iv) it is currently the time when the streaming ML estimation method 200 is scheduled to run; or (v) the streaming ML estimation method 200 should be initiated in response to the occurrence of some other condition. In one embodiment, the streaming ML estimation method 200 is executed by a computer configured with computer-executable instructions to perform the functions of the streaming ML estimation system 100. In one embodiment, the steps of the streaming ML estimation method 200 or other methods herein are executed as a streaming workflow that processes observations as they arrive. After initiation at the start block 205, the streaming ML estimation method 200 proceeds to the processing block 210.

[0033] — Example Streaming ML Estimation Method - Receiving Observations —

[0034] At processing block 210, the streaming ML estimation method 200 receives observations from a stream of observations. The observations arrive as a stream of input to the streaming ML estimation method 200. The observations arrive separated by an interval, such as an interval of the sampling rate.

[0035] In one embodiment, an observation is a vector comprising a plurality of observed values at a particular timestamp. For example, there may be an observed value for each signal in a set of time-series signals. In other words, a stream of observations may be a stream of vectors comprising a time-series database. In one embodiment, the observations are generated by sensors. Thus, in one embodiment, an observation is a vector comprising an observed value for each sensor in a set of sensors. In one embodiment, streaming ML estimation method 200 receives a stream of vectors of observed values from a set of sensors. Thus, for example, a stream of observations is a sequence of vectors in a time-series database of sensor readings.

[0036] In one embodiment, streaming ML estimation system 100 listens to or subscribes to a stream of observations. Streaming ML estimation system 100 detects arriving observations from the stream. In response to detecting the arrival of an observation, streaming ML estimation system 100 ingests or accepts the observation. Streaming ML estimation system 100 places the observation into memory to retain it for subsequent storage, such as in a circular double buffer 120. Receiving observations can be repeated indefinitely while observations continue to arrive from the stream of observations. Receiving observations is performed in real time, so that observations from the stream are retained in memory before being replaced by subsequent observations in the stream.

[0037] Processing block 210 then completes, and the streaming ML estimation method 200 continues at processing block 215. At the completion of processing block 210, the most recently arrived observations from the stream have been ingested and retained in memory for subsequent storage.

[0038] — Example Streaming ML Estimation Method - Loading a Ring Buffer —

[0039] At processing block 215, the streaming ML estimation method 200 loads the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. A circular buffer consisting of the first buffer and the second buffer configured together in a circular configuration may be referred to herein as a circular double buffer.

[0040] Generally speaking, a buffer is an area of memory used to temporarily collect information before processing. For example, a buffer is a data structure. In one embodiment, a buffer includes storage locations for multiple observations. A ring buffer is a form of buffer in which, when the buffer is full and subsequent writes are performed, the buffer continues to add data by overwriting the oldest data. In one embodiment, a ring buffer is a composite buffer consisting of two individual buffers (a first buffer and a second buffer) configured together in a ring configuration. The first buffer and the second buffer are connected end to end in a ring arrangement. In one embodiment, in the ring arrangement of the two individual buffers, when one of the two buffers is full and subsequent writes are performed, data is added to the other buffer by overwriting the oldest data in the other of the two buffers. In one example, this composite buffer arrangement of two buffers as a ring buffer may be referred to herein as a ring double buffer.

[0041] In one embodiment, the first and second buffers that make up the circular double buffer can also be considered "circular" because after the buffer becomes full, old data is overwritten by new data. However, when the end of the buffer is reached, the data in the two independent buffers is not immediately overwritten. Instead, when the end is reached, the write is transferred to the other buffer, and the ML model reads the recently filled buffer to generate the estimate.

[0042] In one embodiment, streaming ML estimation method 200 alternately places newly received observations from the observed stream into one of two buffers. In one embodiment, streaming ML estimation method 200 switches the placement of observation vectors between the two buffers when one of the two buffers becomes full. Specifically, when one of the buffers currently being filled becomes full, the filling or loading of the buffers is switched between the buffers. For example, streaming ML estimation method 200 alternates between placing observations arriving from the input observation stream into a first buffer until the first buffer is full, and placing observations arriving from the input observation stream into a second buffer until the second buffer is full. In other words, streaming ML estimation method 200 switches between buffers in response to the buffer currently receiving newly received observations becoming full. Thus, a first subset of incoming observations is temporarily stored in the first buffer, while a second subset of incoming observations is temporarily stored in the second buffer.

[0043] A buffer becomes full or filled when the number of observations placed in the buffer equals the length of the buffer (or the size of a batch, as discussed below). In other words, the buffer becomes full when an observation is written to the last available position in the buffer. When the buffer becomes full, a complete batch of observations used to generate the ML estimate has been aggregated in the buffer. In one embodiment, the streaming ML estimation method 200 continues loading observations into the first circular buffer until the first circular buffer is full, then into the second circular buffer until the second circular buffer is full, and so on indefinitely while the streaming ML estimation system continues to receive observations from the stream of observations. Thus, the ML model is provided with a continuous sequence of complete batches of observations for processing, thereby removing the input delay for aggregating batches for all but the first batch.

[0044] In one embodiment, an intake pointer indicates which of the two buffers incoming observations will be loaded into. The intake pointer alternates between pointing to the first buffer and the second buffer. The intake pointer points in the opposite direction of the outflow pointer (discussed below in processing block 215), which indicates which buffer the ML model is reading from. In one embodiment, the intake pointer is or includes a variable that stores the address in memory of one of the two buffers. In response to the last position of the second buffer being filled, the intake pointer can be updated to indicate the first buffer (i.e., store the address in memory of the buffer), and in response to the last position of the first buffer being filled, the intake pointer can be updated to indicate the second buffer.

[0045] Each of the two individual buffers that make up the circular double buffer has a buffer length that accommodates batches of observations for processing. This buffer length may also be referred to as the batch size. In one embodiment, the first and second buffers share a common buffer length, i.e., both individual buffers have the same length. Thus, in one embodiment, the first buffer (buffer length R positions) and the second buffer (buffer length also R positions) are together configured as a circular double buffer (length 2R positions). Observations are placed into the individual buffers in the chronological order in which they are received from the stream of observations. When full, the individual buffers contain batches of observations arranged in chronological order for processing by the ML model.

[0046] Observations can be indexed by timestamps. An observation can be a vector of observed values for a set of signals at a timestamp. Thus, a batch of observations is a time series of observations. Thus, each individual buffer has a length equal to the batch size and a width equal to the number of signals. Observations can be viewed as "rows" in the buffer, and signals as "columns" of the buffer. When full, the first and second buffers can each be viewed as a batch-length segment of the time series database delivered by the stream of observations.

[0047] In one example, a timestamp may indicate the time at which an observation was obtained from a sensor, and the timestamp may be included in a data structure for observations arriving from a stream of observations. Alternatively, in another example, the timestamp may indicate the time at which an observation arrived from a stream. As discussed elsewhere herein, an observation may be a vector of observed values for a set of signals at the timestamp. In one embodiment, both the first and buffers discussed herein will contain batches of time series data for processing.

[0048] In one embodiment, loading the observations into the circular double buffer involves placing or writing the observations into the next available location in the circular double buffer. In one embodiment, the observations are written into the next available location and a pointer or other indicator pointing to the next available location is advanced by one location. In one embodiment, the observations are loaded into the circular double buffer in real time such that the observations are immediately placed or written into the circular double buffer in response to the observations being received. For example, loading the observations in real time is accomplished with a maximum delay that is the maximum delay allowed for loading the observations that arrive at the sampling rate of the stream of observations.

[0049] In one embodiment, receiving observations from the observed stream and loading them into the circular double buffer in real time is performed by an input handler, such as input handler 115. The input handler is configured to perform one or more functions described with reference to processing blocks 210 and 215. In one embodiment, receiving observations from the observed stream and loading them into the circular double buffer in real time is performed by an auxiliary compute instance. The auxiliary compute instance can execute in parallel with a primary compute instance configured to generate ML estimates, as described below. Thus, in one embodiment, receiving and loading observations into one buffer occurs in parallel or concurrently with generating ML estimates from another buffer.

[0050] In one embodiment, observations are loaded in real time into a circular double buffer comprised of first and second buffers in an end-to-end configuration by writing each observation to the next available location in the circular double buffer in response to the observation being received, and switching between writing to the next available location in one of the first and second buffers and writing to the next available location in the other of the first and second buffers in response to the last available location in the first and second buffers being loaded with an observation. Processing block 215 is then completed, and the streaming ML estimation method 200 continues at processing block 220. At the end of processing block 215, the batch-sized buffer will be filled with observations for subsequent processing to generate ML estimates. In this manner, the streaming ML estimation method 200 converts the stream of observations into sequential batches for subsequent batched generation of estimates.

[0051] — Example Streaming ML Estimation Method — Generating Estimates —

[0052] At processing block 215, the streaming ML estimation method 200 generates an estimate of what the observations are expected to be from the observations in the circular double buffer via the machine learning model. The generation of the estimate alternates between: generating the estimate from the observations in the first buffer in parallel while the second buffer is being loaded; and generating the estimate from the observations in the second buffer in parallel while the first buffer is being loaded.

[0053] Recall that an observation is a vector of observed values for each signal in a set of signals, and an estimate is a vector of estimated values for each signal in the set of signals. In one embodiment, a machine learning model is configured to execute a function that outputs an estimated value for each observed value in the observation based on the other observed values in the observation. The other observed values are observed values for other signals than the signal from which the estimated value was generated. The ML model retrieves the observation, for example, by reading the observation from one of the first or second buffers in the circular double buffer. The ML model reads the observed values as parameters for the function. The ML model then executes the function to generate an estimated value for each signal from the observed values for the other signals. The ML model writes the estimated value for each signal to the position for that signal in the vector of estimated values. In one embodiment, the estimated values so generated by the ML model can be generated as an estimate data structure—a vector of estimated values for each signal in the set of signals.

[0054] Assuming the behavior reported by a signal is normal, the estimate represents what the observations would have expected. The estimated value for each signal is appropriate or within the expected range, given the observed values for the other signals. If the estimated value for a signal differs sufficiently from the observed value (i.e., has a large residual), then that signal may be reporting abnormal behavior in the monitored asset.

[0055] In one embodiment, an ML model is trained based on training observations that represent normal operation to generate an estimate of what the observations are expected to be. The ML model is trained or configured to generate an estimate of what the observations are expected to be based on the observations in the circular double buffer. To train the machine learning model, a supervised learning process is performed on the machine learning model and the training data. The training data can be a set or series of observations where the observed value for the signal at each observation is defined as a normal, nominal, or typical measured value given other signal values at the observation. The observations in the training set represent normal operation of the monitored asset where the monitored asset is operating within appropriate or expected parameters. The observations in the training set can be live data streamed from a stream of observations. The observations in the training set can also be from a historical database of time series signals.

[0056] In a supervised learning process, the parameters of the ML model's function are iteratively adjusted toward producing estimates that are close to the observed values in the training data. The supervised learning process ends after the ML model consistently produces estimates that are sufficiently similar to the observed values in the training data. The trained ML model can then be used to produce estimates based on observations received from the observed stream.

[0057] In one embodiment, the machine learning model is implemented as one or more nonlinear nonparametric (NLNP) regression algorithms for multivariate pattern recognition or anomaly detection. These ML anomaly detection algorithms may include neural networks (such as long short-term memory (LSTM) networks), support vector machines (SVMs), autoassociative kernel regression (AAKR), and similarity-based modeling (SBMs), such as multivariate state estimation techniques (MSET). Therefore, in one embodiment, the ML model is a NLNP model or an MSET model. Furthermore, in one embodiment, the ML model is a multivariate machine learning model that operates on a set of multiple variables as input to generate estimates for the multiple variables.

[0058] In one embodiment, the ML model generates an estimate of what the observations are expected to be from the observations in the first buffer while the second buffer is being loaded from the stream of real-time observations. Furthermore, the ML model generates an estimate of what the observations are expected to be from the observations in the second buffer while the first buffer is being loaded from the stream of real-time observations. In one embodiment, reading or retrieving estimates from alternating sources (the first and second buffers) in the circular double buffer is repeated in a repeating cycle in response to the continuous arrival of new observations from the stream of observations. In this way, the streaming ML estimation method 200 concurrently generates estimates using the ML model from observations from one of the two buffers while one of the buffers is being loaded with observations from the other of the two buffers.

[0059] For the two individual buffers (a first buffer and a second buffer) that make up the circular double buffer, read operations that retrieve observations from the buffers to the ML model for processing are transferred to the buffers when the buffers become full. In one embodiment, the outflow pointer indicates which of the two buffers the batch of observations is being read from for processing. The outflow buffer indicates which buffer the ML model is sourced from for observations to generate estimates. The outflow pointer alternates between pointing to the first buffer and the second buffer. The outflow pointer points opposite the inflow pointer, indicating the source buffer. In one embodiment, the outflow pointer is or includes a variable that stores the address in memory of one of the two buffers. In one embodiment, the outflow pointer switches between buffers in response to the same events as the inflow pointer (discussed with reference to processing block 215 above). In response to the last position of the first buffer being filled, the outflow pointer may be updated to indicate the first buffer (i.e., store the address in memory of the buffer), and in response to the last position of the second buffer being filled, the outflow pointer may be updated to indicate the second buffer.

[0060] In one embodiment, the streaming ML estimation method 200 generates estimates from observations in two buffers (the first and second buffers of a circular double buffer) using a machine learning model executed by a primary compute instance. Furthermore, the streaming ML estimation method 200 uses a secondary compute instance to load the observations into the two buffers. The primary and secondary compute instances can be, for example, discrete virtual machine (VM) instances or discrete cloud containers. The two compute instances share access to memory and / or storage. The circular double buffer is stored in memory accessible to both compute instances.

[0061] In one embodiment, both the primary computing instance and the auxiliary computing instance operate a machine learning application. The machine learning application includes the following functionality: functionality for executing a machine learning model to generate an estimate, functionality for loading observations from a stream of observations into a circular double buffer, and functionality for writing the estimates generated by the ML model to a stream of output estimates. In one embodiment, the primary computing instance executes the functionality of the machine learning application that causes the machine learning model to generate an estimate. In one embodiment, the auxiliary computing instance executes the functionality of the machine learning application that causes the observations to be loaded into the circular double buffer and the functionality of the machine learning application that causes the estimates (and associated data, such as the observations and residuals corresponding to the estimates) to be written to an output stream. In one embodiment, the auxiliary computing instance may also be referred to herein as an input-output handler.

[0062] In one embodiment, the streaming ML estimation method 200 generates a first estimate (a first vector of estimated values) from observations (a vector of observed values) in a first buffer (of a circular double buffer) using a multivariate machine learning model. Furthermore, the streaming ML estimation method 200 generates a second estimate (a second vector of estimated values) from observations (a vector of observed values) in a second buffer (of a circular double buffer) using a multivariate machine learning model. In one embodiment, while the primary compute instance concurrently and in parallel generates ML estimates from previously loaded batches, the secondary compute instance performs loading of observations from the stream into the first and second buffers to generate batches of observations for processing. In one embodiment, the previously loaded batch is loaded into one buffer immediately after the batch is loaded into the other buffer currently being loaded.

[0063] In one embodiment, the streaming ML estimation method 200 switches which of the two buffers in the circular double buffer is being loaded and which is being processed in response to the currently loading buffer becoming full. For example, in parallel with generating a first estimate using the machine learning model, the streaming ML estimation method 200 places observations from the stream into the second buffer until the second buffer is full. In response to the second buffer being full, the streaming ML estimation method 200 generates a second estimate from the observations in the second buffer using the machine learning model. In parallel with generating the second estimate using the machine learning model, the streaming ML estimation method 200 places observations from the stream into the first buffer until the first buffer is full. In response to the first buffer being full, the streaming ML estimation method 200 generates a first estimate from the observations in the first buffer using the machine learning model.

[0064] In one embodiment, this cycle repeats indefinitely while new observations are received from the stream of observations for which no estimates have been generated. The ML model rotates or alternates between obtaining observations for estimation from the first and second buffers in the circular double buffer. That is, a first batch of observations is obtained from the first buffer while the second buffer is being loaded, and a second batch of observations is obtained from the second buffer while the first buffer is being loaded. Thus, while the buffer currently being filled has not yet become full, the streaming ML estimation method 200 uses the machine learning model to generate a first vector of estimates from a vector of observations that are not currently in the buffer being filled.

[0065] In one embodiment, in parallel with the use of the primary compute instance to generate estimates from observed vectors not currently in the buffer being filled, the use of the secondary compute instance to fill the buffer currently being filled and output the estimates is performed. In one embodiment, the circular double buffer thus enables input-output operations to be performed in parallel with the generation of estimates, thereby reducing processing latency.

[0066] Thus, in one embodiment, the streaming ML estimation method 200 generates estimates from observation data from a previously loaded buffer (of the ring buffer) in parallel (concurrently) while one buffer (of the ring buffer) is being loaded with observations from the stream of observations, and then alternates between the two buffers based on which buffer is currently being loaded. Processing block 220 is then completed, and the streaming ML estimation method 200 continues at processing block 225. At the end of processing block 220, while a batch of observations is being loaded, an ML estimate of what the observations should be has been generated for another previously loaded batch of observations. The ML estimate can be written to a stream of output estimates.

[0067] — Example Streaming ML Estimation Method — Writing Estimates to a Stream —

[0068] At processing block 225, the streaming ML estimation method 200 writes the estimates to a stream of estimates in real time as they are generated. In one embodiment, the estimates generated by the ML model are streamed after they are generated.

[0069] As discussed above, in one embodiment, an estimate is a data structure that includes a vector of estimated values for a set of signals and a timestamp or other index. In one embodiment, the estimate may also include other data in the data structure. In one embodiment, the estimate may include both a vector of estimated values for the set of signals and a vector of corresponding observed values for the set of signals, thereby including both the observations and the estimates together. In one embodiment, the estimate may include a vector of estimated values for the set of signals and a vector of corresponding residual values for the set of signals, thereby including both the residuals from the observations and the estimates together. In one embodiment, the estimate may include a vector of estimated values for the set of signals, a vector of corresponding observed signals for the set of signals, and a vector of corresponding residual values for the set of signals. The "corresponding" estimates, observed values, and residuals correspond with respect to a timestamp or other index.

[0070] In one embodiment, a stream of estimates is defined by a streaming function that streams data values. The streaming function monitors a specific location in memory for outputting estimates by the ML model. After being generated by the streaming ML estimation method 200, the estimate data structure generated at processing block 220 is placed into a location in memory that is monitored by the streaming function. The streaming function detects the location where the estimate is written to the memory. In response to detecting that the estimate is written to the location in the memory, the streaming function retrieves the estimate and then transmits the estimate to a destination. The destination can include, for example, a location in memory or storage, another program or function, or a network location. For example, the destination can be an anomaly detection test program or function (e.g., as discussed below) that is used to determine whether the residual between the observation and the estimate is anomalous.

[0071] In one embodiment, estimates are written to the stream one at a time. Estimates are written in real time. In one embodiment, the process of writing estimates to the stream of estimates in real time does not need to wait for one or more subsequent estimates to be generated before starting to write the estimates to the stream. Instead, the estimates are written to the output stream immediately after and in response to their generation, for example.

[0072] In one embodiment, the estimate is written to the stream by the auxiliary compute instance. In one embodiment, the estimate is written to the stream by the auxiliary compute instance in parallel with the generation of the ML estimate by the primary compute instance. For example, the streaming ML estimation method 200 uses the auxiliary compute instance to write the estimate to the stream of output estimates. In one embodiment, the auxiliary compute instance writes the estimate generated while the primary compute instance is generating a subsequent estimate from observations stored in a circular double buffer. The auxiliary compute instance writes the estimate generated by the primary compute instance while the primary compute instance is concurrently generating the subsequent estimate. Because the writing of the estimate to the stream is handled concurrently, in parallel with the generation of the subsequent estimate, the latency of writing the estimate to the stream is hidden by the computation latency of the estimate generation. This further reduces end-to-end latency.

[0073] In one embodiment, the streaming ML estimation method 200 monitors the output of the ML model for generating estimates; then, upon detecting that an estimate has been generated by the ML model, the method immediately retrieves the estimate and transmits it to a destination (such as a downstream anomaly detection test function or program) while the ML model is operating to produce a subsequent estimate. Processing block 225 is then complete, and the streaming ML estimation method 200 proceeds to end block 230, where the method 200 is complete.

[0074] Upon completion of streaming ML estimation method 200, ML estimates for observations received from the stream have been generated and streamed in real time, reducing the end-to-end latency introduced by the streaming ML estimation process. Due to the steps described herein using a circular double buffer, streaming ML estimation method 200 generates the estimates in the stream with minimal latency incurred by input and output operations, with the majority of latency incurred by the actual generation of the ML estimates. In one embodiment, this enables even earlier detection and warning of anomalies when they occur, compared to other methods for generating streaming estimates.

[0075] — Further Embodiments of Streaming ML Estimation Methods —

[0076] In one embodiment, before initiating the parallelized processing of the ML estimation and input-output activities discussed above (using a circular double buffer arrangement of first and second buffers), the method determines whether the serial processing of input, ML estimation, and output is too slow. Thus, in one embodiment, before loading the observations into the circular buffer in real time, the end-to-end latency of the serial processing of the observations to the estimation is measured, and if the end-to-end latency of the serial processing is too high, then the processing switches to parallelized processing.

[0077] In one embodiment, before loading the observations into the ring buffer in real time, the streaming ML estimation method 200 additionally measures the end-to-end delay of serially loading the observations, generating estimates from the observations, and writing the estimates to the estimated stream. The streaming ML estimation method 200 then compares the end-to-end delay to a threshold for maximum acceptable delay. Next, in response to the threshold being met, the streaming ML estimation method 200 switches to loading the observations into the ring buffer.

[0078] If the threshold is met (i.e., the end-to-end delay is above the threshold), then serial processing is too slow (the end-to-end delay is too high), and parallel processing should take over to ensure sufficient throughput to keep up with the stream of observations. If the threshold is not met (i.e., the end-to-end delay is below the threshold), then serial processing is fast enough (the end-to-end delay is low enough) to allow sufficient throughput. In this case, the streaming ML estimation method 200 will continue to process the observations serially. The streaming ML estimation method 200 can occasionally recheck to determine whether the threshold is met to indicate a shift to parallel processing.

[0079] In one embodiment, the operations of the streaming ML estimation method 200 are parallelized across two compute instances: a primary compute instance for generating the ML estimation, and a secondary compute instance for handling the streaming input and streaming output operations. In one embodiment, both the primary and secondary compute instances are virtual machines. In one embodiment, both the primary and secondary compute instances are instances of a containerized machine learning application. Both the primary and secondary compute instances have access to the first and second buffers.

[0080] Thus, in one embodiment, the streaming ML estimation method 200 additionally instantiates a primary compute instance. The primary compute instance is configured to generate estimates via the ML model. The streaming ML estimation method 200 also additionally instantiates an auxiliary compute instance. The auxiliary compute instance is configured to load observations into a circular buffer and write estimates to a stream of estimates. The streaming ML estimation method 200 places the circular buffer (including the first and second buffers) into a location(s) in memory that is accessible to both the primary compute instance and the auxiliary compute instance. Additional details regarding compute instances are discussed above with reference to processing block 220 and below, for example, with reference to Figure 3 Let’s discuss.

[0081] An estimate at an index point (such as a timestamp) is generated from an observation for that index point. In this way, an estimate corresponds to an observation, where the index point of the estimate is the index point of the observation. In one embodiment, when writing to the estimated stream, the estimate is accompanied by the corresponding observation from which the estimate was generated. For example, an estimate can be generated by an ML model as a data structure that includes both the estimate and the observation for one timestamp. If an observation is a vector that includes the observed value for each signal in a set of signals at a timestamp, then the estimate can be a vector that includes both the observed value for each signal in the set of signals at that timestamp and the estimated value for each signal in the set of signals. Therefore, in one embodiment, the streaming ML estimation method 200 also writes the observation corresponding to the estimate to the estimated stream along with the estimate.

[0082] The residual or difference between the observation and the estimate can be used to detect anomalies in the observation. In one embodiment, the ML model can generate the residual by subtracting the estimate from the corresponding observation. The estimate, observation, and residual correspond, where the index point of the residual is the index point of the observation and the estimate. In one embodiment, when writing to the estimated stream, the estimate is accompanied by the corresponding residual between the estimate and the corresponding observation from which the estimate was generated. For example, the estimate can be generated by the ML model as a data structure that includes both the estimate and the residual for a timestamp. If the estimate is a vector that includes the estimated value for each signal in the set of signals at that timestamp, then the estimate can also include the residual value between the estimated value and the observed value for each signal in the set of signals at that timestamp. Therefore, in one embodiment, the streaming ML estimation method 200 also writes the residual corresponding to the estimate to the estimated stream together with the estimate.

[0083] As discussed above, with reference to processing blocks 210 and 220, in one embodiment, the observations and estimates are vectors comprising values for a plurality of signals. And, in one embodiment, the machine learning model is a multivariate ML model that generates an estimate for each signal based on observed values of other signals different from the signal for which the estimate is generated. Thus, in one embodiment, the observations are vectors comprising observed values for each signal in a set of signals. And, the estimates are vectors comprising estimated values for each signal in the set of signals. And, the machine learning model is a multivariate machine learning model. Generating the estimates (as discussed at processing block 220) then further comprises generating an estimate for a signal in the set of signals based on observed values for other signals in the set of signals different from the signal. Thus, an estimate can be generated for each signal in the set from the observed values for the other signals in the set.

[0084] As discussed herein, an ML model is used to detect anomalies in a stream of observations based on the residuals between the corresponding observations and estimates. An observed value in a signal is anomalous if the observed value deviates from the estimated value in a manner that satisfies a threshold test for detecting an anomaly. In one embodiment, upon detecting an anomaly, an alert can be generated indicating the presence of the anomaly. In one embodiment, the streaming ML estimation method 200 also generates an alert indicating the detection of an anomaly in response to one or more residuals between one of the observations and an estimate for that one of the observations satisfying a threshold.

[0085] In one embodiment, to generate an alert, the streaming ML estimation method 200 analyzes the residuals between the observation and the estimate to determine whether the observation is anomalous using a test for detecting anomalies. In one embodiment, the residuals can be parsed from the estimate data structure (the location of which is included in the estimate data structure as discussed above) or calculated from the estimate and the observation. In one embodiment, the test for detecting anomalies is a sequential probability ratio test (SPRT). The SPRT calculates the cumulative sum of the log-likelihood ratios of each successive residual between the observed value for a signal and the estimated value for the signal, and compares the cumulative sum to a threshold value indicating an anomalous deviation. When the threshold is crossed, an anomaly is detected.

[0086] In one embodiment, after detecting an anomaly in one or more observed values in an observation, an alert may be generated to inform a user or other system that an anomaly exists. In one embodiment, the alert is an electronic message. The alert may be generated by writing an electronic message, which is then presented and the generated alert is transmitted for subsequent display or other action. The alert may be configured to be displayed in a graphical user interface. For example, an alert may be used to inform an operator of a monitored asset that the asset is behaving abnormally or in an unexpected manner. The alert may be configured to trigger a request (such as a REST request) to initiate some other function. For example, an alert may be used to initiate an automatic adjustment or shutdown of the operation of a monitored asset, or to initiate an automatic maintenance request regarding the monitored asset. The alert may be generated and presented in real time after the anomaly is detected, to enable a quick response to the detected anomaly.

[0087] In one embodiment, the streaming ML estimation method 200 determines the buffer length, for example, by accepting user input for the buffer length or automatically determining the buffer length experimentally. In one embodiment, a batch size (the batch size being the length of the first and second buffers) can be provided to the streaming ML estimation method 200, and the buffer lengths can be adjusted to accommodate the batch size. The batch size (or buffer length) can be a user-entered value. In one embodiment, the batch size is measured in the number of observation vectors or rows. In one embodiment, the batch size is the number of observation vectors to be stored in one buffer of the circular double buffer before the buffer becomes full and the process alternates between processing the buffer and storing the vectors in the other buffer of the circular double buffer. In one embodiment, a user or administrator enters the buffer length into the system, for example, in a user interface for configuring the streaming ML estimation system 100. After providing the buffer length, the streaming ML estimation method 200 configures the first and second buffers to have the length (i.e., the number of available rows) equal to the provided buffer length. Therefore, in one embodiment, the streaming ML estimation method 200 also accepts input for the buffer length and configures the first and second buffers to accept the same number of observations as the buffer length before becoming full.

[0088] In one embodiment, the batch size or buffer length can be automatically determined experimentally for a given use case. As used herein, the term "use case" refers to a specific configuration of the streaming rate and the number of signals (the number of observed values per observation). In one embodiment, the buffer length is automatically identified so that the streaming ML estimation method 200 keeps up with the stream of observations when generating the ML estimate of the number of signals. Therefore, in one embodiment, the streaming ML estimation method 200 also automatically identifies the buffer length; and configures the first and second buffers to accept the same number of observations as the buffer length before being full.

[0089] In one embodiment, to automatically identify the buffer length (batch size) for a use case, the processing latency for each batch used to execute the streaming ML estimation method 200 is repeatedly measured for different batch sizes on a set of test data. The test data includes the number of signals for the use case. Observations from the test data are streamed to the streaming ML estimation system 100 at the streaming rate for the use case. If the total processing latency for a batch is less than the time it takes to stream all observations in the batch, then processing can keep up with the stream in real time. Therefore, in one embodiment, the processor identifies batch sizes for which the processing latency for each batch is less than the time it takes to stream all observations in the batch (streaming rate × batch size). In one embodiment, the measured processing latency for each batch should be lower than the time it takes to stream all observations in the batch by a certain margin (such as 20%) to allow for possible variations in processing latency.

[0090] In one embodiment, one of the identified batch sizes is automatically selected as the buffer length. In one embodiment, the processor automatically selects the identified batch size at which the processing latency for each batch is minimized. Note that processing latency is dominated by read and write latency at smaller batch sizes and by ML estimation latency at larger batch sizes, as discussed below. In the event that the minimum processing latency per batch for the use case remains higher than the time it takes to stream all observations in the batch, a faster compute configuration may be indicated. In one embodiment, the above process of automatically identifying batch sizes (buffer lengths) may be repeated for progressively faster compute configurations until the compute configuration and buffer length are determined to be able to keep up with streaming. Additional details regarding identifying batch sizes / buffer lengths are discussed below.

[0091] In one embodiment, the ML model is trained before being used to generate the estimate. In one embodiment, the ML model is trained based on training observations representing normal operation to generate an estimate of what the observations are expected to be. Additional details about training the ML model are discussed above with reference to processing block 220.

[0092] In one embodiment, the ML model is a Multivariable State Estimation Technique (MSET) model, as discussed above in processing block 215 .

[0093] In one embodiment, one or more non-transitory computer-readable media have computer-executable instructions (also referred to as program instructions) stored thereon. The computer-executable instructions are configured to cause one or more computers to perform operations including the operations of the streaming ML estimation method 200 (or other methods described herein) when the computer-executable instructions are executed.

[0094] In one embodiment, a computing system includes one or more computers configured by computer-executable instructions to perform operations including the operations of the streaming ML estimation method 200 (or other methods described herein).

[0095] In one embodiment, a computer program product includes a computer program. The computer program, when executed by at least a processor of a computer, causes the computer to perform operations including the operations of the streaming ML estimation method 200 (or other methods described herein). As an example, the computer program may include one or more computer-executable instructions that cause a computer to perform the operations.

[0096] —Additional Context and Examples—

[0097] Multivariate machine learning (ML) can be used to monitor time series signals for automated anomaly detection. Generally speaking, ML anomaly detection services analyze data in static batches, such as data from a previously recorded data history database. ML anomaly detection can be extended to a real-time streaming mode so that new time series streams from live sensors can be analyzed in real-time to detect anomalies (and trigger alerts) at the earliest possible time. In one example, ML anomaly detection can be extended to a real-time streaming mode by performing cyclic batch processing (such as by repeatedly calling an ML function) to sequentially analyze batches (e.g., frames or buffers) of customer data.

[0098] One challenge in converting static batch-style ML analysis is to estimate the throughput and latency for large-scale ingestion and processing of data in streaming mode. In one embodiment, streaming mode should be configured to maintain real-time performance for a given number of signals and a given sampling rate selected for the data to be monitored. Otherwise, when the throughput volume of data (i.e., the number of signals at the sampling rate) reaches a point where processing cannot maintain real-time performance, the system crashes and / or latency accumulates in an unbounded manner. Either of these situations can cause timeout exceptions in the upstream data ingestion hardware / software interface, or even cause data to be discarded at the data ingestion interface.

[0099] The challenge of estimating end-to-end real-time processing latency in the analysis of ML anomaly detection algorithms is complicated by the fact that ML computational cost grows nonlinearly with the number of signals being monitored, but rather linearly with the product of the buffer size for the data being analyzed and the sampling rate. Furthermore, there is a read latency component for the processing delay of filling the buffer, and a write latency component for the processing delay of writing the results to the output stream.

[0100] For example, compare a simple throughput architecture (or "pipeline") for ML processing, a simple throughput architecture (or "pipeline") where processing time is generally directly proportional, and a simple throughput architecture (or "pipeline") where no processing occurs. In a throughput architecture for ML processing, an incoming data stream fills a buffer for processing, and when the buffer is full, the buffer is processed by the ML algorithm, and then the results are written to an output stream. In a simple throughput architecture without processing, no ML computation is performed when the buffer becomes full, and data is simply moved from input to output.

[0101] In the example simple throughput architecture with no processing, estimating the end-to-end latency is straightforward: there is a read latency to fill the buffer, and a write latency to the output buffer. If data is moved through the pipeline in large batches (e.g., a buffer length of 100,000 rows), then the read & write latency is moderate. However, if data is moved through the pipeline in small batches (e.g., a buffer length of 10 rows), then the read and write latency components dominate the overall latency for the sustained throughput of the streaming time series. However, for non-ML applications (where batch computations are not performed on the contents of the buffer), estimating the overall streaming latency is straightforward. The cumulative latency is a simple linear function of the buffer length (or number of rows per block of data).

[0102] In an example simple throughput architecture for ML processing, where ML computations are performed on the contents of a buffer, decomposing the end-to-end latency becomes significantly more challenging. For ML anomaly detection algorithms for time series, the computational cost increases quadratically with the number of signals in the buffer and linearly with the number of rows in the buffer. This presents a complex, nonlinear tradeoff between the number of signals, the number of rows, the buffer length, and the compute shape (e.g., various types of central processing units (CPUs) and graphics processing units (GPUs)), each of which significantly alters the processing latency, or time, required for ML to process the contents of the buffer.

[0103] Therefore, the challenge of estimating end-to-end real-time processing latency for ML anomaly detection techniques (such as multivariate state estimation techniques (MSET), neural networks, and support vector machines (SVM)) is compounded by the fact that ML computational cost increases quadratically with the number of signals, but increases linearly with the product of the buffer size and sampling rate for the data being analyzed. There is still a read latency component for filling the buffer, and a write latency component for writing the results to the output stream. But now estimating the end-to-end throughput and cumulative latency (to ensure that the streaming computation can keep up with the real-time ingestion of time series data) becomes significantly more challenging. Furthermore, determining the end-to-end throughput latency of streaming ML anomaly detection techniques is not practically amenable to brute force trials (such as by incrementally increasing the number of signals and / or sampling rate until the overall throughput can no longer keep up with real time), simply because changing the number of signals or sampling rate for a new use case changes the bottleneck point at which the streaming / processing architecture will be unable to keep up with the incoming ingestion rate.

[0104] The processing time (or computational cost) for streaming data processing that is not associated with input and output can range from very small, near-zero processing time due to passing data only through I / O channels, to processing time that is directly proportional to the volume of data (and therefore easy to measure and scale), to complex, nonlinear growth for ML anomaly detection processing. Examples of very small, near-zero processing costs include straight-through streaming of video and archiving data from a streaming source to a storage system. Examples of linearly scalable processing costs include taking the square root of all quantities passing through a pipeline, data encryption / decryption, converting units of time series, scaling data statistics, and language conversion. Existing techniques for estimating end-to-end latency are straightforward for both simple systems with no processing costs for "data in motion" and for use cases where processing costs scale linearly with the "volumetric throughput" of the data (in which case the processing overhead for small data streams is easily measured and scaled linearly to the bandwidth capacity of the architecture).

[0105] Estimating the end-to-end throughput and latency of time series data streams using ML anomaly detection is challenging, especially for multivariate pattern recognition. Examples of multivariate tools / techniques include Multivariate State Estimation Techniques (MSET) (including Oracle's specialized MSET2), neural networks (including long short-term memory (LSTM) networks), and support vector machines (SVMs).

[0106] The challenge with these multivariate anomaly detection techniques is that the computational cost (and therefore processing latency) scales nonlinearly with the volume of time series data. For example, the computational cost for a given data buffer scales both quadratically with the number of signals in the buffer and linearly with the number of rows in the buffer. Thus, for example, the time to process a buffer with N columns of signals (from sensors) and M rows or vectors (timestamps) of time series data will be significantly different from the time to process an equally sized buffer consisting of M columns and N rows (assuming M is not equal to N).

[0107] Note that the signal used for anomaly detection use cases can involve a mix of univariate and multivariate (meaning correlated) signals. The preprocessing step for anomaly detection identifies univariate signals from multivariate signals and separates the univariate signals for separate processing. This preprocessing analysis is a one-time upstream and offline analysis and is therefore not part of the estimated throughput and latency of streaming multivariate ML anomaly detection.

[0108] In one embodiment, this document describes two processing modes for multivariate ML anomaly detection. The first processing mode uses a single circular buffer for multivariate ML anomaly detection on streaming signals. The second processing mode uses a double circular buffer for multivariate ML anomaly detection on streaming signals. Each method can be used, for example, in cloud, cloud edge, or local server implementations of multivariate ML anomaly detection on streaming signals.

[0109] Generally speaking, in multivariate ML time series analysis, calculations operate on buffers of data. In one embodiment, the buffer of data is an array of time series numbers. In the array of time series numbers, columns represent sensors and rows represent timestamps or observations (although the opposite convention can also be used, where columns represent timestamps and rows represent sensors). An example might be a signal from an asset with 30 sensors, and 1000 time-stamped observations would be loaded into a buffer with 30 columns and 1000 rows at a time. Continuous buffering of streaming data is generally used to handle streaming data.

[0110] In one embodiment, the buffer is a circular buffer. The circular buffer is configured to temporarily store streaming data. In the circular buffer, the buffer is configured to be filled with new incoming data and to be cleared after the ML anomaly detection analysis is completed as the results are written out. The buffer length (number of rows) remains the same for each application of the ML algorithm. In the simple example above with 1000 rows, the circular buffer is filled with the incoming stream of 1000 rows of ingested data, these 1000 rows of data are analyzed by the ML algorithm to identify the residuals between the observed and estimated values, and then 1000 rows of results (corresponding to these 1000 rows of data) are written to the output stream.

[0111] In single-ring buffer streaming throughput processing, for each batch of data, read, ML analysis, and write processing occur in linear order. There is a "read latency" at the beginning of the operation, which is the time it takes to fill the buffer. Read latency increases linearly with the size of the buffer (number of signals multiplied by the sampling rate). There is an ML analysis latency, which is the time it takes to perform ML analysis on the contents of the buffer. ML analysis latency scales nonlinearly with the size of the data in the buffer. Specifically, ML analysis latency scales quadratically with the number of signals (columns) but scales linearly with the number of observation vectors (rows). There is a write latency, which is the time it takes to write the results to the output stream. Like read latency, write latency scales linearly with the size of the data in the buffer. Write latency can be greater than read latency because the output result may have more columns than the input stream. For example, ML analysis outputs the raw, unprocessed time series signal along with the computed results for that signal, meaning the output stream has twice as many columns as the input stream. However, the output write latency still scales linearly with the amount of data in the buffer.

[0112] Note that for a given use case (where the number of sensors is fixed and cannot be changed), when using a single-buffer configuration, the overall throughput latency can vary significantly depending on the chosen buffer length (i.e., the number of rows per buffer). Depending on the length (in rows) of the ring buffer chosen for the batch to be processed by ML, when there is a single ring buffer, the latency factorization involves three important components: read latency (proportional to the number of signals multiplied by the number of input rows), processing (ML analysis) latency (scaling with the square of the number of signals multiplied by the number of input rows), and write latency (proportional to the number of signals multiplied by the number of output rows). For any given use case, there can be a wide range of end-to-end latency, which depends largely on the ring buffer length. The ring buffer length can be user-configurable. In a single-buffer framework for ML anomaly detection in streaming data, the end-to-end latency can be accurately parameterized and evaluated as a function of the window width (number of rows).

[0113] Performing ML analysis on large sensor arrays in real time can be challenging or exceed the capabilities of available computer processing specifications. Note that, at the time of this writing, a single commercial aircraft can have over 75,000 sensors, while a modern oil refinery or a medium-sized data center can have over 1,000,000 sensors each. This paper presents a circular double-buffer, alternating pointer framework for ML anomaly detection in streaming data as an alternative to using a single-buffer framework in use cases where end-to-end throughput latency exceeds the maximum latency requirements of large-scale real-time processing applications. The circular double-buffer approach hides latency for all (or nearly all) read / write latencies, as discussed in further detail elsewhere in this paper.

[0114] The latency in a typical streaming application using a single ring buffer approach is characterized by three latency components for each buffer of data: read time, processing (ML analysis) time, and write time. Thus, for sequential single-buffer processing, after B buffers, the total end-to-end latency L T It will be B buffers multiplied by the read latency L R , processing delay L P and write latency L W (L T =B×(L R +L P +L W )). The total delay is linearly proportional to the buffer size or the dimensions of the buffer in terms of the number of measurements (rows) and the number of signals. However, the processing delay L P It increases quadratically with the number of signals in the buffer that are processed by the ML model.

[0115] Note that these three latency components are generally applicable to ML anomaly detection techniques, including MSET, neural networks, and SVMs. ML technique-specific hyperparameters (such as the numvecs (number of vectors) hyperparameter in MSET2) are independent of latency. Therefore, in one embodiment, the latency calculation and circular double-buffer streaming shown and described herein are compatible with any ML technique.

[0116] As an alternative to the single-buffer approach where read and write latencies are cumulatively stacked with the processing latency for ML anomaly detection, the circular double-buffer approach hides read and write latencies, thereby maximizing the content of each buffer. In one embodiment, two compute instances are launched for the circular double-buffer approach, so each compute instance has a dedicated buffer. The two dedicated buffers can each be accessed by the two compute instances. The two dedicated buffers can be configured together end-to-end in a circular configuration to form a circular double buffer.

[0117] In one embodiment, under the circular double buffer approach, both the ingest pointer and the outgest pointer alternate between two dedicated buffers. The ingest pointer indicates which of the two dedicated buffers making up the circular double buffer is currently being loaded with incoming observations, for example, by a secondary instance. The outgest pointer indicates which of the two dedicated buffers is currently being read by the ML model. The ingest pointer and the outgest pointer indicate opposite buffers in the two dedicated buffers making up the circular double buffer. While the primary instance is performing computations, the secondary instance is loading the next batch of data.

[0118] Using this approach, for the same B batches of data in the previous example, there is only one read latency L R , plus B processing delays L P , plus a write delay L W (L T =L R +(B×L P )+L W ). Thus, B occurrences of the combined read and write latency are reduced from the total end-to-end latency. Thus, the circular double buffer, alternating pointer approach hides the latency for read and write operations by having auxiliary compute instances perform the read and write operations in parallel. Figure 3 Additional details regarding the operation of a circular double buffer method using alternating pointers are shown.

[0119] Figure 3 A scope diagram 300 for single-buffer operation and a scope diagram 305 for circular double-buffer operation using an alternating pointer that switches between buffers are illustrated. Scope diagrams 300 and 305 illustrate latency broken down into components and distributed among compute instances. Scope diagram 300 for single-buffer operation illustrates the order in which read, process, and write operations are performed using one compute instance 310 in a single-buffer configuration. Scope diagram 305 for circular double-buffer operation illustrates the order in which read, process, and write operations are performed using a primary compute instance 315 and a secondary compute instance 320 in a circular double-buffer configuration. Read, process, and write operations are shown alongside compute timeline 325. Read blocks (shown with diagonal hatching) take approximately a first amount of compute time to complete. Process blocks (shown without hatching) take approximately a second amount of compute time to complete. Write blocks (shown with cross-hatching) take approximately a third amount of compute time to complete.

[0120] In a single-buffer configuration using one compute instance 310, the one compute instance 310 reads a first batch of data at a read block 330, adding a first amount of computation time to the end-to-end latency. The one compute instance 310 then processes the first batch of data at a processing block 331 to generate an ML estimate for the first batch, adding a second amount of computation time to the end-to-end latency. The one compute instance 310 then writes the ML estimate for the first batch of data to the output stream at a write block 332, adding a third amount of computation time to the end-to-end latency. This process is then repeated for the second and subsequent batches of data. For example, by the one compute instance 310 reading the second batch at a read block 333, the first amount of computation time is again added to the end-to-end latency. By the one compute instance 310 processing the ML estimate for the second batch at a processing block 334, the second amount of computation time is again added to the end-to-end latency. And, by the one compute instance 310 writing the estimate to the stream at a write block 335, the third amount of computation time is again added to the end-to-end latency.

[0121] In a circular double-buffer configuration using a primary compute instance 315 and a secondary compute instance 320, secondary compute instance 320 reads a first batch of data at read block 340, adding a first amount of computation time to the end-to-end latency. Primary compute instance 315 then processes the first batch of data at processing block 341 to generate an ML estimate for the first batch of data, adding a second amount of computation time to the end-to-end latency. While primary compute instance 315 is generating the ML estimate at processing block 341, secondary compute instance 320 is reading a second batch of data at read block 342. Although read block 342 consumes the first amount of computation time, the first amount of computation time for read block 342 is not added to the end-to-end latency because read block 342 is executed by secondary compute instance 320 concurrently with primary compute instance 315 executing processing block 341.

[0122] When the primary compute instance 315 has completed the estimate for the first batch, the second batch has been loaded by the secondary compute instance 320. The primary compute instance 315 then processes the second batch of data at processing block 343 to generate the ML estimate, adding a second amount of computation time to the end-to-end latency. While the primary compute instance 315 is generating the ML estimate for the second batch at processing block 343, the secondary compute instance 320 is (i) writing the estimate for the first batch of data to the output stream at writing block 344, and (ii) reading the third batch of data at reading block 345. Although the read block 345 consumes the first amount of computation time and the write block 344 consumes the third amount of computation time, the first and third amounts of computation time for the read block 345 and the write block 344 are not added to the end-to-end latency because these read and write blocks are executed by the secondary compute instance 320 while the primary compute instance 315 is generating the estimate for the second batch at block 343.

[0123] In one embodiment, read and write operations performed by the auxiliary compute instance 320 can be intermixed, although shown as discrete boxes in the diagram 305 of the circular double buffer operation. In one embodiment, write operations for a batch are not delayed until after the batch of ML estimates is processed, although shown as after the batch is processed in the diagram 305 of the circular double buffer operation. For example, as discussed above, individual estimates are written to the stream of estimates by the auxiliary compute instance 320 immediately after they are generated by the primary compute instance 315. Thus, some individual estimates from a batch can be written to the stream of estimates concurrently with the generation of other estimates in the batch.

[0124] This process continues in a similar manner for subsequent batches of data. For example, auxiliary compute instance 320 reads subsequent batches of data and writes the estimates while primary compute instance 315 generates ML estimates for the current batch of data. This adds only a second amount of computation time to the end-to-end latency for each batch until after the last batch. After the estimates are generated for the last batch of data, the final write operation for that last batch is performed by auxiliary compute instance 320, adding a third amount of computation time to the end-to-end latency.

[0125] Figure 4 An example graph 400 of total latency versus number of signals 405 for streaming ML anomaly detection using a single-buffer processing configuration is shown. Total latency versus number of signals 405 is plotted against a first axis 410 showing the number of signals in the buffer and a second axis 415 showing the total latency for streaming ML anomaly detection. Note that the line for total latency versus number of signals 405 is not smooth because the amount of time required for read, write, and load operations is somewhat random as the time taken for ML anomaly detection accumulates.

[0126] To demonstrate the latency improvement using the circular double buffer approach, Figure 4 The streaming use case presented in for the single-buffer approach is re-run using the ring-double-buffer approach, as Figure 5 Note that when measuring delay, only the resulting total end-to-end delay is measured, since the goal is to minimize the total end-to-end delay.

[0127] Figure 5 An example graph 500 shows total latency versus number of signals 505 for streaming ML anomaly detection using a circular double buffer processing configuration. Total latency versus number of signals 505 is plotted against a first axis 510 showing the number of signals in the buffer and a second axis 515 showing the total latency for streaming ML anomaly detection. Figure 4 and Figure 5As can be seen, the total latency is reduced by an order of magnitude. For example, the total latency is reduced by approximately 90%. In addition, the line 505 of total latency versus number of signals using the circular dual-buffer processing configuration is smoother than the line 405 of total latency versus number of signals using the single-buffer configuration. The line 505 of total latency versus number of signals is smoother because the randomly timed read, write, and load operations are not performed in conjunction with the ML anomaly detection processing. For example, in one embodiment, the read, write, and load operations are not performed by the first compute instance that also performs the ML anomaly detection processing. Instead, these important buffer loads are hidden. For example, in one embodiment, the read, write, and load operations are instead performed by the second compute instance that does not perform the ML anomaly detection processing.

[0128] While switching from a processing loop utilizing a single circular buffer to a processing loop utilizing a circular double buffer as described herein significantly reduces end-to-end latency for streaming ML anomaly detection, latency is generally not further reduced by adding more buffers. Figure 3 As discussed, the reason for the significant speedup from single-buffer processing to circular double-buffer processing comes from using a circular double buffer to generate ML, parallelizing the read and write latencies for filling and releasing the buffers with the processing latency estimated for generating the ML. After hiding the latency by changing from a single buffer to a circular double buffer in the processing loop, when the combined read and write latencies are less than the processing latency (which is typically the case), no further gains will be gained by using 3, 4, ... N circular buffers. When the combined read and write latencies are longer than the processing latency, additional buffers can further reduce the processing latency, but in practice, the combined read and write latencies are typically less than the processing latency.

[0129] — Advantages of some choices —

[0130] In one embodiment, the systems and methods described herein propose a circular double-buffer configuration for multivariate anomaly detection on a stream of real-time values. In one embodiment, the circular double-buffer configuration hides read / write latency and achieves higher throughput and lower end-to-end computation latency than a single-buffer configuration.

[0131] In one embodiment, the systems and methods described herein enable transitioning batch multivariate ML anomaly detection to a real-time streaming architecture.

[0132] In one example, this paper proposes an accurate framework for isolating and analyzing the component contributors to end-to-end latency in single-ring buffer ML processing and ring-double buffer ML processing.

[0133] — Cloud or Enterprise Implementation —

[0134] In one embodiment, the system (such as streaming ML estimation system 100) is a computing / data processing system that includes a computing application or a collection of distributed computing applications for access and use by other client computing devices in communication with the system over a network. In one embodiment, streaming ML estimation system 100 is a component of a time series data service configured to collect, provide, and perform operations on time series data. The application and computing system can be implemented as or configured to operate with a cloud-based network computing system, Infrastructure as a Service (IaaS), Platform as a Service (PAaS), or Software as a Service (SAaS) architecture, or other types of networked computing solutions. In one embodiment, the system provides at least one or more of the functionalities disclosed herein and a graphical user interface for accessing and operating these functionalities. In one embodiment, streaming ML estimation system 100 is a centralized server-side application that provides at least the functionalities disclosed herein and is accessible to many users via computing devices / terminals that communicate with the computers of streaming ML estimation system 100 (acting as one or more servers) over a computer network. In one embodiment, the streaming ML estimation system 100 may be implemented by a server or other computing device configured with hardware and software that implements the functions and features described herein.

[0135] In one embodiment, the components of the streaming ML estimation system 100 can be implemented as a collection of one or more software modules executed by one or more computing devices specifically configured for such execution. In one embodiment, the components of the streaming ML estimation system 100 are implemented on one or more hardware computing devices or hosts interconnected via a data network. For example, the components of the streaming ML estimation system 100 can be executed by network-connected computing devices of one or more computing hardware form factors, such as a central processing unit (CPU), or general-purpose form factors, intensive input / output (I / O) form factors, graphics processing unit (GPU) form factors, and high-performance computing (HPC) form factors.

[0136] In one embodiment, the components of streaming ML estimation system 100 communicate with each other via electronic messages or signals. These electronic messages or signals can be configured as calls to functions or procedures that access features or data of the components, such as, for example, application programming interface (API) calls. In one embodiment, these electronic messages or signals are sent between hosts in a format compatible with Transmission Control Protocol / Internet Protocol (TCP / IP) or other computer networking protocols. Components of streaming ML estimation system 100 can (i) generate or compose electronic messages or signals to issue commands or requests to another component, (ii) transmit messages or signals to other components of streaming ML estimation system 100, (iii) parse the content of received electronic messages or signals to identify commands or requests that the component can execute, and (iv) automatically run or execute the commands or requests in response to identifying the commands or requests. The electronic messages or signals can include queries against a database. The queries can be written and executed in a query language compatible with the database and can be executed in a runtime environment compatible with the query language.

[0137] In one embodiment, a remote computing system can access information or applications provided by streaming ML estimation system 100, for example, via a web interface server. In one embodiment, the remote computing system can send requests to streaming ML estimation system 100 and receive responses from it. In one example, access to information or applications can be performed using a web browser on a personal computer or mobile device. In one example, communications exchanged with streaming ML estimation system 100 can take the form of Remote Representational State Transfer (REST) requests, for example, using JavaScript Object Notation (JSON) as a data exchange format, or Simple Object Access Protocol (SOAP) requests to and from an XML server. REST or SOAP requests can include API calls to components of streaming ML estimation system 100.

[0138] —Software Module Embodiment—

[0139] Generally speaking, software instructions are designed to be executed by one or more appropriately programmed processors accessing memory. These software instructions may include, for example, computer executable code and source code that can be compiled into computer executable code. These software instructions may also include instructions written in an interpreted programming language (such as a scripting language).

[0140] In complex systems, such instructions may be arranged into program modules, wherein each such module performs a specific task, process, function, or operation. The entire collection of modules may be controlled or coordinated by an operating system (OS) or other form of organizational platform.

[0141] In one embodiment, one or more of the components described herein are configured as modules stored in a non-transitory computer-readable medium. The module is configured with stored software instructions that, when executed by at least a processor accessing a memory or storage device, cause a computing device to perform (one or more) corresponding functions as described herein.

[0142] —Computing Device Embodiment—

[0143] Figure 6 An example computing system 600 is illustrated that includes an example computing device configured and / or programmed as a special-purpose computing device having one or more of the example systems and methods described herein, and / or equivalents. The example computing device may be a computer 605 that includes at least one hardware processor 610, a memory 615, and input / output ports 620 operatively connected via a bus 625. In one example, the computer 605 may include streaming ML estimation logic 630 configured to generate a stream of ML estimates in real time from an observed stream using a circular double buffer, similar to the example of FIG. Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 and Figure 5 The logic, systems, and methods shown and described.

[0144] In various examples, the logic 630 may be implemented in hardware, a non-transitory computer-readable medium 637 storing instructions, firmware, and / or a combination thereof. While the logic 630 is illustrated as a hardware component attached to the bus 625, it should be appreciated that in other embodiments, the logic 630 may be implemented in the processor 610, stored in the memory 615, or stored on the disk 635.

[0145] In one embodiment, the logic 630 or computer is a means (e.g., structure: hardware, non-transitory computer-readable media, firmware) for performing the described actions. In some embodiments, the computing device can be a server operating in a cloud computing system, a server configured in a software as a service (SaaS) architecture, a smartphone, a laptop computer, a tablet computing device, etc.

[0146] The components may be implemented, for example, as an ASIC programmed to facilitate the real-time generation of a stream of ML estimates from a stream of observations using a circular double buffer. The components may also be implemented as stored computer-executable instructions that are presented to the computer 605 as data 640, which is temporarily stored in the memory 615 and then executed by the processor 610.

[0147] The logic 630 may also provide means (eg, hardware, non-transitory computer-readable medium storing executable instructions, firmware) for generating a stream of ML estimates in real-time from a stream of observations using a circular double buffer.

[0148] Generally describing an example configuration of computer 605, processor 610 can be a variety of processors, including dual microprocessors and other multi-processor architectures. Memory 615 can include volatile memory and / or non-volatile memory. Non-volatile memory can include, for example, ROM, PROM, etc. Volatile memory can include, for example, RAM, SRAM, DRAM, etc.

[0149] The storage disk 635 can be operatively connected to the computer 605 via an input / output (I / O) interface (e.g., card, device) 645 and an input / output port 620, for example, controlled by at least an input / output (I / O) controller 647. The disk 635 can be, for example, a magnetic disk drive, a solid-state drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, a memory stick, etc. Additionally, the storage disk 635 can be a CD-ROM drive, a CD-R drive, a CD-RW drive, a DVD ROM, etc. The memory 615 can store, for example, processes 650 and / or data 640. The storage disk 635 and / or the memory 615 can store an operating system that controls and allocates resources of the computer 605.

[0150] The computer 605 can interact with, control, and / or be controlled by input / output (I / O) devices via an input / output (I / O) controller 647, an I / O interface 645, and input / output ports 620. The input / output devices may include, for example, one or more displays 670, a printer 672 (such as an inkjet printer, a laser printer, or a 3D printer), an audio output device 674 (such as a speaker or a headset), a text input device 680 (such as a keyboard), a cursor control device 682 for pointing and selecting input (such as a mouse, a trackball, a touch screen, a joystick, a pointing stick, an electronic stylus, an electronic writing tablet), an audio input device 684 (such as a microphone or an external audio player), a video input device 686 (such as a video camera and a still camera, or an external video player), an image scanner 688, a video card (not shown), a disk 635, a network device 655, and the like. The input / output ports 620 may include, for example, a serial port, a parallel port, and a USB port.

[0151] The computer 605 can operate in a network environment and, therefore, can be connected to a network device 655 via the I / O interface 645 and / or the I / O port 620. Through the network device 655, the computer 605 can interact with the network 655. Through the network, the computer 605 can be logically connected to a remote computer 665. Networks with which the computer 605 can interact include, but are not limited to, LANs, WANs, and other networks.

[0152] In one embodiment, a computer may be connected to a sensor 690 via an I / O port 620 or a network 660 to receive information about the physical state of a monitored machine, device, system, or facility (collectively, an "asset"). In one embodiment, the sensor 690 is configured to monitor physical phenomena occurring in or around an asset. An asset generally includes any type of machine or facility having components that perform a measurable activity. In one embodiment, the sensor 690 may be operably connected to or secured to an asset, or otherwise configured to detect and monitor physical phenomena occurring in or around an asset. The sensor 690 may be a network-connected sensor for monitoring any type of physical phenomenon. The network connection between the sensor 690 and the network 660 may be wired or wireless.

[0153] In one embodiment, the computer 605 is configured with logic (such as a software module) to collect readings from the sensors 690 and store these readings as observations in a time series data structure (such as a time series database). In one embodiment, the computer 605 passively receives sensor telemetry readings that are actively transmitted by the sensors 690. For example, the sensor telemetry readings can be transmitted from the sensors 690 to the computer 605 in real-time. In one embodiment, the time series database is stored in a buffer, as discussed above. In one embodiment, the computer 605 polls the sensors 690 to retrieve the sensor telemetry readings. In one embodiment, the computer 605 receives one or more databases of previously collected observations of the sensors 690, such as from the storage device 635 or from the remote computer 665.

[0154] —Definitions and Other Examples—

[0155] Any action or function described or claimed herein is not performed by the human mind. Any interpretation that any action or function can be performed by the human mind is inconsistent and contrary to this disclosure.

[0156] In another embodiment, the described methods and / or their equivalents may be implemented using computer-executable instructions. Thus, in one embodiment, a non-transitory computer-readable / storage medium is configured with computer-executable instructions of a stored algorithm / executable application that, when executed by a (one or more) machine, causes the (one or more) machine (and / or associated components) to perform the described method. Example machines include, but are not limited to, processors, computers, servers operating in a cloud computing system, servers configured in a software-as-a-service (SaaS) architecture, smartphones, and the like. In one embodiment, a computing device is implemented with one or more executable algorithms configured to perform any of the disclosed methods.

[0157] In one or more embodiments, the disclosed methods or their equivalents are performed by any of the following: computer hardware configured to perform the method; or, computer instructions embodied in a module stored in a non-transitory computer-readable medium, wherein the instructions are configured as an executable algorithm that is configured to perform the method when executed by at least a processor of a computing device.

[0158] Although for the purpose of simplifying the description, the method illustrated in the figure is shown and described as a series of blocks of an algorithm, it should be appreciated that these methods are not limited by the order of the blocks. Some blocks may appear in a different order than shown and described and / or appear simultaneously with other blocks. Moreover, the example method can be implemented using fewer blocks than all the illustrated blocks. Blocks can be combined or divided into multiple actions / components. In addition, additional and / or alternative methods can employ additional actions not illustrated in the blocks.

[0159] The following includes definitions of selected terms used herein. The definitions include various examples and / or forms of components that fall within the scope of the terms and can be used for implementation. The examples are not intended to be limiting. Both singular and plural forms of the terms may be included within the definitions.

[0160] References to "one embodiment," "an embodiment," "an example," "an example," etc. indicate that the embodiment(s) or example(s) described herein may include a particular feature, structure, characteristic, property, element, or limitation, but not every embodiment or example must include the particular feature, structure, characteristic, property, element, or limitation. Furthermore, repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, but may refer to the same embodiment.

[0161] As used herein, a "data structure" is an organization of data stored in a memory, storage device, or other computerized system in a computing system. A data structure can be, for example, any of a data field, a data file, a data array, a data record, a database, a data table, a graph, a tree, a linked list, and the like. A data structure can be formed from and contain many other data structures (e.g., a database includes many data records). Other examples of data structures are possible according to other embodiments.

[0162] As used herein, "computer-readable medium" or "computer storage medium" refers to a non-transitory medium that stores instructions and / or data that are configured to perform one or more of the disclosed functions when executed. In some embodiments, data can be used as instructions. Computer-readable media can take the form of, but is not limited to, non-volatile media and volatile media. Non-volatile media can include, for example, optical disks, magnetic disks, etc. Volatile media can include, for example, semiconductor memories, dynamic memories, etc. Common forms of computer-readable media can include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, other magnetic media, application-specific integrated circuits (ASICs), programmable logic devices, compact disks (CDs), other optical media, random access memories (RAMs), read-only memories (ROMs), memory chips or cards, memory sticks, solid-state storage devices (SSDs), flash drives, and other media that computers, processors, or other electronic devices can utilize to work. If each type of media is selected for implementation in one embodiment, it can include stored instructions of an algorithm that is configured to perform one or more of the disclosed and / or claimed functions.

[0163] As used herein, "logic" refers to a component implemented using computer or electrical hardware, a non-transitory medium with instructions of stored executable applications or program modules, and / or a combination of these to perform any of the functions or actions disclosed herein, and / or to cause the functions or actions from another logic, method, and / or system to be performed as disclosed herein. Equivalent logic can include firmware, a microprocessor programmed with an algorithm, discrete logic (e.g., ASIC), at least one circuit, analog circuit, digital circuit, programmed logic device, a memory device containing instructions of the algorithm, etc., any of which can be configured to perform one or more of the disclosed functions. In one embodiment, logic can include one or more gates, a combination of gates, or other circuit components configured to perform one or more of the disclosed functions. In the case of describing multiple logics, it is possible to integrate multiple logics into one logic. Similarly, in the case of describing a single logic, it is possible to distribute the single logic between multiple logics. In one embodiment, one or more of these logics are corresponding structures associated with performing the disclosed and / or claimed functions. Choosing which type of logic to implement can be based on the desired system conditions or specifications. For example, if higher speed is considered, hardware will be selected to implement the function. If lower cost is a consideration, stored instructions / executable applications will be chosen to implement the functionality.

[0164] An "operable connection," or a connection through which entities are "operably connected," is a connection in which signals, physical communications, and / or logical communications can be sent and / or received. An operable connection may include a physical interface, an electrical interface, and / or a data interface. An operable connection may include different combinations of interfaces and / or connections sufficient to allow operable control. For example, two entities may be operably connected to pass signals to each other directly or through one or more intermediate entities (e.g., a processor, an operating system, logic, a non-transitory computer-readable medium). Logical and / or physical communication channels may be used to create an operable connection.

[0165] As used herein, a "user" includes, but is not limited to, one or more persons, computers or other devices, or a combination of these.

[0166] Although the disclosed embodiments have been illustrated and described in considerable detail, it is not intended to restrict or in any way limit the scope of the appended claims to such detail. Of course, it is not possible to describe every contemplated combination of components or methods for the purpose of describing various aspects of the subject matter. Therefore, the present disclosure is not limited to the specific details or illustrative examples shown and described. Therefore, the present disclosure is intended to cover changes, modifications, and variations that fall within the scope of the appended claims.

[0167] To the extent that the term "including" or "comprising" is employed in the detailed description or the claims, it is intended to be inclusive in a manner similar to how the term "comprising" is interpreted when employed as a transitional word in the claims.

[0168] To the extent the term "or" is employed in the detailed description or claims (e.g., A or B), it is intended to mean "A or B or both." When the applicants intend to indicate "only A or B but not both," then the phrase "only A or B but not both" will be used. Thus, the use of the term "or" herein is inclusive, not exclusive.

Claims

1. A computer-implemented method for generating an estimated flow from an observed flow in real time, the method comprising: Receive observations from an observed stream; loading the observations into a ring buffer in real time, wherein the ring buffer comprises a first buffer and a second buffer configured together in a ring configuration; generating, by the machine learning model, estimates of what is expected from the observations in the circular buffer, wherein generating the estimates alternates between: generating the estimates in parallel from the observations in the first buffer while the second buffer is being loaded, and generating the estimates in parallel from the observations in the second buffer while the first buffer is being loaded; as well as Estimates are written to the estimates stream in real time as they are generated.

2. The computer-implemented method of claim 1 , further comprising generating an alert that an anomaly was detected in response to one or more residuals between one of the observations and one of the estimates for the one of the observations satisfying a threshold.

3. The computer-implemented method of any one of claims 1-2, further comprising: Measuring the end-to-end latency of serially loading observations, generating estimates from them, and writing the estimates to an estimated stream; comparing the end-to-end delay to a threshold value of maximum acceptable delay, and In response to a threshold of maximum acceptable latency being met, a switch is made to loading the observations into the ring buffer.

4. The computer-implemented method of any one of claims 1 to 3, further comprising: Instantiating a master computing instance, wherein the master computing instance is configured to generate an estimate using the ML model; instantiating an auxiliary compute instance, wherein the auxiliary compute instance is configured to load observations into a ring buffer and write estimates to an estimated stream; as well as The circular buffer is placed in a location in memory that is accessible to both the primary and secondary compute instances.

5. The computer-implemented method of any one of claims 1 to 4, further comprising: Write the observations corresponding to the estimates together with the estimates to the estimates stream; as well as Writes the residuals corresponding to the estimate along with the estimate to the estimates stream.

6. The computer-implemented method of any one of claims 1 to 5, where observations is a vector containing the observed values for each signal in the set of signals, where estimates is a vector comprising estimated values for each signal in the set of signals, where the machine learning model is a multivariate machine learning model, and Generating estimates also includes: An estimated value for the one signal in the set of signals is generated based on observed values for other signals in the set different from the one signal.

7. A computer-implemented method as described in any of claims 1-6, further comprising training a machine learning model based on training observations representing normal operation to generate an estimate of what the observations are expected to be.

8. The computer-implemented method of any one of claims 1 to 7, further comprising: Determine the buffer length; as well as The first buffer and the second buffer are configured to accept a number of observations equal to the buffer length before becoming full.

9. A computer-implemented method as described in any of claims 1-8, wherein the machine learning model is a multivariable state estimation technology model.

10. One or more non-transitory computer-readable media having computer-executable instructions stored thereon, wherein the computer-executable instructions are configured to cause one or more computers to perform operations including the operations of any one of claims 1-9 when executing the computer-executable instructions.

11. A computing system comprising one or more computers, wherein the computing system is configured by computer-executable instructions to perform operations including the operations of any one of claims 1-9.

12. A computer program product comprising a computer program which, when executed by at least a processor of a computer, causes the processor to perform operations including the operations of any one of claims 1 to 9.

13. One or more non-transitory computer-readable media comprising computer-executable instructions stored thereon for generating an estimated flow from an observed flow in real time, the computer-executable instructions, when executed by at least a processor of a computer, causing the computer to: Receive observations from an observed stream; loading the observations into a ring buffer in real time, wherein the ring buffer comprises a first buffer and a second buffer configured together in a ring configuration; generating, by the machine learning model, estimates of what is expected from the observations in the circular buffer, wherein generating the estimates alternates between: generating the estimates in parallel from the observations in the first buffer while the second buffer is being loaded, and generating the estimates in parallel from the observations in the second buffer while the first buffer is being loaded; as well as Estimates are written to the estimates stream in real time as they are generated.

14. The non-transitory computer-readable medium of claim 12, further comprising instructions that, when executed by at least the processor, cause the processor to: Measuring the end-to-end latency of serially loading observations, generating estimates from them, and writing the estimates to an estimated stream; comparing the end-to-end delay to a threshold value of maximum acceptable delay, and In response to a threshold of maximum acceptable latency being met, a switch is made to loading the observations into the ring buffer.

15. A computing system comprising: at least one processor coupled to at least one memory; a non-transitory computer-readable medium comprising computer-executable instructions stored thereon for generating an estimated flow from an observed flow in real time, the instructions, when executed by at least the processor, causing the computing system to: Receive observations from an observed stream; loading the observations into a ring buffer in real time, wherein the ring buffer comprises a first buffer and a second buffer configured together in a ring configuration; generating, by the machine learning model, estimates of what is expected from the observations in the circular buffer, wherein generating the estimates alternates between: generating the estimates in parallel from the observations in the first buffer while the second buffer is being loaded, and generating the estimates in parallel from the observations in the second buffer while the first buffer is being loaded; as well as Estimates are written to the estimates stream in real time as they are generated.

16. The computing system of claim 14, wherein the instructions further cause the computing system to: Instantiating a master computing instance, wherein the master computing instance is configured to generate an estimate using the ML model; instantiating an auxiliary compute instance, wherein the auxiliary compute instance is configured to load observations into a ring buffer and write estimates to an estimated stream; as well as The circular buffer is placed in a location in memory that is accessible to both the primary and secondary compute instances.

Images