A circular buffer for generating machine learning estimates of streaming observations in real time.

Abstract

Systems, methods, and other embodiments relating to generating a stream of ML estimates from a stream of observations in real time using a circular dual buffer are described. In an exemplary method, observations are received from a stream of observations. The observations are read into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. A machine learning model generates expected observation estimates from the observations in the circular buffer. Estimate generation alternates between generating estimates from the observations in the first buffer in parallel with reading the second buffer, and generating estimates from the observations in the second buffer in parallel with reading the first buffer. The estimates are written to a stream of estimates in real time as they are generated.

Description

[Background technology]

[0001] background Sensors for a wide variety of physical phenomena may be attached to machines, devices, systems, or facilities (collectively referred to as “assets”). These sensors generate data about physical phenomena occurring within or around the assets. The data generated by the sensors may be monitored or analyzed by computers. [Overview of the Initiative]

[0002] overview In one embodiment, one or more non-temporary computer-readable media are presented. The non-temporary computer-readable media includes stored computer-executable instructions for generating a stream of estimates from a stream of observations in real time. When executed at least by the computer's processor, the computer-executable instructions cause the computer to perform operations or steps of a method for generating a stream of estimates from a stream of observations in real time. These instructions cause the computer to receive observations from the stream of observations. These instructions cause the computer to read the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. These instructions cause the computer, by a machine learning model, to generate expected estimates of observations from the observations in the circular buffer. The generation of estimates alternates between (1) generating estimates from the observations in the first buffer in parallel while the second buffer is being read, and (2) generating estimates from the observations in the second buffer in parallel while the first buffer is being read. Furthermore, these instructions cause the computer to write the estimates to the estimate stream in real time as the estimates are generated.

[0003] In one embodiment, a computer method is presented for generating a stream of estimates from a stream of observations in real time. This method includes receiving observations from a stream of observations. This method includes reading the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. This method includes generating expected observation estimates from the observations in the circular buffer using a machine learning model. Estimate generation alternates between (1) generating estimates from the observations in the first buffer in parallel while the second buffer is being read, and (2) generating estimates from the observations in the second buffer in parallel while the first buffer is being read. This method also includes writing the estimates to a stream of estimates in real time as they are generated.

[0004] In one embodiment, a computing system is presented. The computing system includes at least one processor connected to at least one memory and one or more non-temporary computer-readable media. The non-temporary computer-readable media includes stored computer-executable instructions for generating a stream of estimates from a stream of observations in real time. When executed by a processor with access to at least memory, the computer-executable instructions cause the computing system to perform an operation or step of a method for generating a stream of estimates from a stream of observations in real time. These instructions cause the computing system to receive observations from the stream of observations. These instructions cause the computing system to read the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. These instructions cause the computing system, by a machine learning model, to generate expected observation estimates from the observations in the circular buffer. The generation of estimates involves alternately repeating the following steps: (1) generating estimates from the observations in the first buffer while the second buffer is being read, and (2) generating estimates from the observations in the second buffer while the first buffer is being read, in parallel. These instructions also cause the computing system to write the estimates to an estimate stream in real time as they are generated.

[0005] The accompanying drawings incorporated herein and constituting part thereof illustrate various systems, methods, and other embodiments of the present disclosure. It should be understood that the element boundaries illustrated in the drawings (e.g., boxes, groups of boxes, or other shapes) represent one embodiment of the boundary. In some embodiments, one element may be implemented as multiple elements, or multiple elements may be implemented as a single element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component, and vice versa. Furthermore, elements may not be depicted to scale. [Brief explanation of the drawing]

[0006] [Figure 1] This figure shows one embodiment of a streaming ML estimation system that uses a circular double buffer to generate a stream of ML estimates in real time from a stream of observations. [Figure 2] This figure shows one embodiment of a streaming ML estimation method that involves generating a stream of ML estimates in real time from a stream of observations using a circular double buffer. [Figure 3] This figure shows the range of operation for a single buffer and the range of operation for a circular double buffer that uses alternating pointers to switch buffers. [Figure 4] This figure shows an illustrative graph of total latency against the number of signals for streaming ML anomaly detection using a single-buffer processing configuration. [Figure 5] This figure shows an illustrative graph of the total delay time against the number of signals for streaming ML anomaly detection using a cyclic double buffer processing configuration. [Figure 6]This figure illustrates one embodiment of a computing system comprising the exemplary systems and / or methods disclosed. [Modes for carrying out the invention]

[0007] Detailed explanation This specification describes systems, methods, and other embodiments for generating a stream of machine learning (ML) estimates in real time from a stream of observations using a circular double buffer. In one embodiment, a streaming ML estimation system parallelizes I / O activities with ML estimation activities to reduce end-to-end latency. For example, the ML estimation system stores observations in two opposing buffers connected together in a circular manner, and retrieves observations from these buffers, allowing I / O latency to occur concurrently with (and be masked by) ML processing latency.

[0008] In one embodiment, a streaming ML estimation system receives an incoming stream of observations. The observations in the stream are placed in a circular buffer in real time as they are received. Estimates are generated by the ML model from the observations in the circular buffer. The ML model generates estimates of the expected observations. While the observations are placed in the first buffer of the circular buffer, the ML model generates estimates from the second buffer of the circular buffer, and vice versa. The estimates are written to an output stream in real time as they are generated. The stream of estimates may be used in conjunction with the stream of observations to detect anomalies in the stream of observations.

[0009] The acts or functions described or claimed herein are not performed by human thought. Any interpretation that any act or function can be performed by human thought is inconsistent with and contrary to this disclosure.

[0010] -Definition- As used herein, the terms “stream” and “streaming” refer to the transmission, use, and reception of data as a steady, continuous flow (e.g., at a certain sampling rate) that allows some of the data to be used while it is being received.

[0011] As used herein, the term “real-time” means substantially immediate action that keeps pace with the throughput of the data stream. In other words, a real-time process occurs within a maximum delay time that does not reduce the throughput of the data stream.

[0012] As used herein, the term “delay” refers to the delay or amount of time required for processing and / or transmitting data. As used herein, the term “end-to-end delay” refers to the delay that occurs from the start of a process to the end of a process. For example, the end-to-end delay may be from the arrival of observations in a streaming ML estimation system to writing the estimates of the observations to the output stream (as described below).

[0013] As used herein, the term “time series” refers to a data structure in which a series of data points (such as observations or sampled values) are indexed in chronological order. In one embodiment, the data points of a time series may be indexed using an index such as a timestamp and / or observation number. As used herein, the terms “time series signal” and “time series” are synonymous.

[0014] As used herein, the term "vector" refers to a data structure that includes a set of data points (such as observed values or sampling values) from multiple time series at a single specific index, such as a timestamp and / or an observation number.

[0015] As used herein, the term "time series database" refers to a data structure that includes one or more time series that commonly share an index (such as a series of timestamps, positions, or observation numbers). As an example, a time series may be considered a "column" of a time series database, and a vector may be considered a "row" of a time series database.

[0016] As used herein, the term "residual" refers to the difference between a value (such as a measurement value, an observed value, a sampling value, or a resampling value) and an estimated value, a reference value, or a predicted value of the expected value. In one embodiment, the residual is a positive or negative value. In another embodiment, the residual is an absolute value or magnitude. In one embodiment, a residual time series or a time series of residuals refers to a time series composed of residual values between a time series of values and a time series of expected values. Also, in one embodiment, a residual vector or a vector of residuals refers to a vector composed of residual values between a vector of values and a vector of expected values.

[0017] -Exemplary Streaming ML Estimation System- FIG. 1 shows one embodiment of a streaming ML estimation system 100 related to generating a stream of ML estimates from a stream of observed values in real time using a circular double buffer. The streaming ML estimation system 100 includes components for generating a stream of estimates 105 from a stream of observed values 110 in real time. The components of the streaming ML estimation system 100 include an input handler 115, a circular double buffer 120, a machine learning model 125, and an output handler 130. In one embodiment, each of these components 115, 120, 125, and 130 of the streaming ML estimation system 100 may be implemented as software executed by computer hardware. For example, the components 110, 115, 120, 125, and 130 may be implemented as one or more software modules, routines, or services that communicate with each other to perform the functions of the components.

[0018] The input handler 115 is configured to receive the stream of observed values 110 as an input and to read the individual observed values 135 received from the stream of observed values 110 into the circular double buffer 120. The circular double buffer 120 is configured to store the observed values 135 from the input handler and to provide the observed values 140 to the machine learning model 125. The machine learning model 125 is configured to generate an estimate 145 of the observed values 140 from the expected circular buffer. The output handler 130 is configured to write the estimate 145 in real time as an output to the stream of estimates 105 upon generation of the estimate 145.

[0019] The circular dual buffer 120 includes a first buffer 150 and a second buffer 155 configured together in a circular configuration. The circular configuration of the first buffer 150 and the second buffer 155 links the ends of the buffers in a loop. When one of the buffers 150, 155 becomes full and a subsequent write operation is performed, the write is performed on the other buffer of buffers 150, 155, overwriting the oldest data. The generation of the estimate 145 by the machine learning model 125 alternates between generating the estimate 145 from the observation 140 in the first buffer 150 (as shown in reference number 160) in parallel (or simultaneously) while the second buffer 155 is being read (as shown in reference number 165), and generating the estimate 145 from the observation 140 in the second buffer 155 (as shown in reference number 170) in parallel (or simultaneously) while the first buffer 150 is being read (as shown in reference number 175).

[0020] In one embodiment, the machine learning model 125 is configured to receive training data 180, which is a set of observations representing normal, nominal, or standard measurements of observations. The machine learning model 125 is configured to perform an initial training operation to tune itself to generate estimates that match the training data 180.

[0021] In one embodiment, the machine learning model 125 is executed by a primary compute instance, and the input handler 115 and output handler 130 are executed by an ancillary compute instance. A circular buffer 120, which includes a first buffer 150 and a second buffer 155, is located in memory or storage, accessible from both the primary and ancillary compute instances.

[0022] In one embodiment, the streaming ML estimation system 100 implements a cyclic double-buffer architecture for parallelized ML estimation and input / output processing. Further details relating to the streaming ML estimation system 100 are presented below. In one embodiment, the operation of the streaming ML estimation system 100 is described with reference to the exemplary streaming ML estimation method 200 shown in Figure 2. In one embodiment, the delay times resulting from the operation of the streaming ML estimation system 100 are described with reference to the delay time decomposition range charts 300, 305 shown in Figure 3, and to the graphs 400, 500 shown in Figures 4 and 5, respectively.

[0023] -Exemplary Streaming ML Estimation Method- Figure 2 shows one embodiment of the Streaming ML Estimation Method 200, which relates to generating a stream of ML estimates from a stream of observations in real time using a circular double buffer. In summary, in one embodiment, the Streaming ML Estimation Method 200 receives observations from a stream of observations. The Streaming ML Estimation Method 200 reads the observations into a circular buffer in real time. The circular buffer includes a first buffer and a second buffer configured together in a circular configuration. The Streaming ML Estimation Method 200 generates expected observation estimates from the observations in the circular buffer using a machine learning model. Estimation generation alternates between generating estimates from the observations in the first buffer in parallel while the second buffer is being read, and generating estimates from the observations in the second buffer in parallel while the first buffer is being read. The Streaming ML Estimation Method 200 writes the estimates to a stream of estimates in real time as the estimates are generated.

[0024] In one embodiment, the streaming ML estimation method 200 starts in a start block 205 in response to the streaming ML estimation system determining one or more of the following: (i) an incoming stream of observations has been detected; (ii) a command has been received to perform the streaming ML estimation method 200 on the stream of observations; (iii) a user or administrator of the streaming ML estimation system 100 has started the streaming ML estimation method 200; (iv) it is now the time when the streaming ML estimation method 200 is scheduled to be performed; or (v) the streaming ML estimation method 200 should be started in response to the occurrence of any other condition. In one embodiment, a computer configured by computer-executable instructions to perform the functions of the streaming ML estimation system 100 executes the streaming ML estimation method 200. In one embodiment, the steps of the streaming ML estimation method 200 or other methods herein are performed as a streaming workflow that processes observations as they arrive. After starting in the start block 205, the streaming ML estimation method 200 proceeds to a process block 210.

[0025] -Example of a streaming ML estimation method- Receiving observations- In process block 210, the streaming ML estimation method 200 receives observations from a stream of observations. The observations arrive as a stream of input to the streaming ML estimation method 200. The observations arrive separated by intervals such as the sampling rate interval.

[0026] In one embodiment, the observations are a vector containing multiple observations at a specific timestamp. For example, there may be observations for each signal in a set of time-series signals. In other words, the stream of observations may be a stream of vectors that constitute a time-series database. In one embodiment, the observations are generated by a sensor. Therefore, in one embodiment, the observations are a vector containing observations for each sensor in a set of sensors. In one embodiment, the streaming ML estimation method 200 receives a stream of vectors of observations from a set of sensors. Therefore, for example, the stream of observations is a sequence of vectors in a time-series database of sensor readings.

[0027] In one embodiment, the streaming ML estimation system 100 listens to or subscribes to a stream of observations. The streaming ML estimation system 100 detects incoming observations from the stream. In response to the detection of an incoming observation, the streaming ML estimation system 100 captures or accepts the observation. The streaming ML estimation system 100 places the observation in memory and holds it for subsequent storage, for example, storage in a cyclic double buffer 120. Observation reception may be repeated indefinitely as long as observations continue to arrive from the stream of observations. Observation reception is performed in real time to hold observations from the stream in memory before observations in the stream are replaced by subsequent observations.

[0028] Subsequently, process block 210 completes, and the streaming ML estimation method 200 continues in process block 215. Upon completion of process block 210, the most recently arrived observations from the stream are captured and held in memory for subsequent storage.

[0029] -Example of a streaming ML estimation method- Reading a circular buffer- In process block 215, the streaming ML estimation method 200 reads observations into a cyclic buffer in real time. The cyclic buffer includes a first buffer and a second buffer configured together in a cyclic configuration. A cyclic buffer consisting of a first buffer and a second buffer configured together in a cyclic configuration may be referred to herein as a cyclic double buffer.

[0030] Generally, a buffer is a region of memory used to temporarily collect information prior to processing. For example, a buffer is a data structure. In one embodiment, a buffer contains storage locations for multiple observations. A circular buffer is a form of buffer in which, when the buffer becomes full and a subsequent write is performed, the buffer continues to add data by overwriting the oldest data. In one embodiment, a circular buffer is a composite buffer consisting of two separate buffers (a first buffer and a second buffer) configured together in a circular configuration. The first buffer and the second buffer are connected end to end in a circular arrangement. In one embodiment, in the circular arrangement of two separate buffers, when one of the two buffers becomes full and a subsequent write is performed, data is added to the other buffer of the two buffers by overwriting the oldest data in the other buffer. In one example, this composite arrangement of two buffers as a circular buffer may be referred to herein as a circular double buffer.

[0031] In one embodiment, the first and second buffers constituting a circular dual buffer can also be considered "circular" in that older data is overwritten by newer data after the buffer is full. However, the data in these two separate buffers is not immediately overwritten when the end of the buffer is reached. Instead, writes are transferred to the other buffer when the end is reached, and the newly filled buffer is read by the ML model to generate estimates.

[0032] In one embodiment, the streaming ML estimation method 200 alternately places newly received observations from the observation stream into one of two buffers. In one embodiment, the streaming ML estimation method 200 switches the placement of the observation vector between the two buffers when one of the two buffers becomes full. Specifically, buffer filling, or buffer reading, is swapped between buffers when one of the currently filled buffers becomes full. For example, the streaming ML estimation method 200 alternately places observations arriving from the observation input stream into the first buffer until the first buffer is full, and places observations arriving from the observation input stream into the second buffer until the second buffer is full. In other words, the streaming ML estimation method 200 switches buffers in response to newly received observations filling the buffer currently receiving them. Thus, a first subset of incoming observations is temporarily stored in the first buffer, and a second subset of incoming observations is temporarily stored in the second buffer.

[0033] The buffer is full or filled when a number of observations equal to the buffer length (or batch size as described below) are placed in the buffer. In other words, the buffer is full when an observation is written to the last available position in the buffer. When the buffer is full, a complete batch of observations for generating an ML estimate has been collected in the buffer. In one embodiment, the streaming ML estimation method 200 continues to read observations into a first circular buffer until the first circular buffer is full, and continues to read observations into a second circular buffer until the second circular buffer is full, indefinitely in repeating cycles while the streaming ML estimation system continues to receive observations from a stream of observations. Thus, the ML model is provided with a continuous sequence of complete batches of observations for processing, thereby eliminating the input delay time for collecting batches for all batches except the first batch.

[0034] In one embodiment, an intake pointer indicates which of two buffers an incoming observation is being read into. The intake pointer alternately points to the first buffer and then to the second buffer. The intake pointer points opposite to an outflow pointer (described in process block 215 below) that indicates the buffer being read by the ML model. In one embodiment, the intake pointer is a variable that stores the address of one of the two buffers in memory, or includes such a variable. The intake pointer may be updated to point to the first buffer (i.e., store the address of the first buffer in memory) in response to the last position of the second buffer being filled, and the intake pointer may be updated to point to the second buffer in response to the last position of the first buffer being filled.

[0035] Each of the two separate buffers that make up a circular double buffer has a buffer length that accommodates a batch of observations for processing. This buffer length is sometimes called the batch size. In one embodiment, the first and second buffers share a common buffer length, i.e., both of the two individual buffers are the same length. Thus, in one embodiment, the first buffer (at a position of buffer length R) and the second buffer (also at a position of buffer length R) are configured together to form a circular double buffer (at a position of length 2R). The observations are placed in the individual buffers in chronological order of reception from the stream of observations. When an individual buffer is full, it contains a batch of observations arranged in chronological order for processing by the ML model.

[0036] Observations may be indexed using timestamps. An observation may be a vector of observations of a set of signals at the timestamp. Thus, a batch of observations is a time series of observations. Thus, each individual buffer has a length of batch size and a width of number of signals. Observations may be considered as "rows" in the buffer, and signals may be considered as "columns" in the buffer. The first and second buffers, when full, may each be considered a segment of the batch length of the time series database being delivered by the stream of observations.

[0037] In one example, the timestamp may indicate the time the observation was acquired from the sensor, and the timestamp may be included in the data structure of the observations arriving from the stream of observations. Or, in another example, the timestamp may indicate the time the observation arrived from the stream. As described elsewhere in this specification, the observation may be a vector of observations of a set of signals at the timestamp. In one embodiment, the first and second buffers described herein both contain batches of time-series data for processing.

[0038] In one embodiment, reading an observation into a cyclic double buffer includes placing the observation at the next available position in the cyclic double buffer or writing it. In one embodiment, the observation is written to the next available position, and the pointer or other indicator of the next available position is advanced by one position. In one embodiment, reading an observation into a cyclic double buffer in real time causes the observation to be rapidly placed in or written to the cyclic double buffer in response to the observation being received. For example, reading an observation in real time is completed with a maximum delay that allows the reading of observations arriving at the sampling rate of the observation stream.

[0039] In one embodiment, receiving observations from a stream of observations and reading the observations into a cyclic dual buffer in real time is performed by an input handler, such as input handler 115. The input handler is configured to perform one or more functions as described with reference to process blocks 210 and 215. In one embodiment, receiving observations from a stream of observations and reading the observations into a cyclic dual buffer in real time is performed by an auxiliary computation instance. The auxiliary computation instance may run in parallel with a primary computation instance configured to generate ML estimates, as described below. Thus, in one embodiment, receiving observations and reading them into one buffer occurs in parallel with, or simultaneously with, the generation of ML estimates from another buffer.

[0040] In one embodiment, observations are read in real time into a cyclic double buffer, which consists of first and second buffers connected end to end, by writing each observation to the next available position in a cyclic double buffer in response to the reception of an observation, and by switching between writing to the next available position in one of the first and second buffers and writing to the next available position in the other of the first and second buffers in response to the last available position in one buffer being read of an observation. After process block 215 is completed, the streaming ML estimation method 200 continues in process block 220. At the end of process block 215, a batch-sized buffer is filled with observations for subsequent processing to generate ML estimates. In this way, the streaming ML estimation method 200 transforms the stream of observations into a series of batches for subsequent batch-by-batch estimation.

[0041] -Exemplary Streaming ML Estimation Method-Estimate Generation- In process block 215, the streaming ML estimation method 200 generates estimated values ​​of expected observations from observations in a cyclic double buffer using a machine learning model. The generation of estimates alternates between generating estimates from observations in the first buffer in parallel while the second buffer is being read, and generating estimates from observations in the second buffer in parallel while the first buffer is being read.

[0042] Recall that observations are a vector of per-signal observations in a set of signals, and estimates are a vector of per-signal estimates in a set of signals. In one embodiment, a machine learning model is configured to run a function that outputs an estimate for each observation in an observation, based on other observations in the observation. Other observations are observations of signals other than the signal for which estimates are being generated. The ML model retrieves observations, for example, by reading them from one of the first or second buffers in a cyclic double buffer. The ML model reads the observations as parameters for a function. The ML model then runs the function to generate per-signal estimates from the observations of the other signals. The ML model writes the per-signal estimates to the signal locations in the estimate vector. In one embodiment, the estimates thus generated by the ML model may be generated as an estimates data structure (a vector of per-signal estimates in a set of signals).

[0043] The estimate represents the expected observed value, provided that the behavior reported by the signal is normal behavior. The estimate for each signal is appropriate or within the expected range, given the observed values ​​of other signals. If the estimate and observed values ​​for a signal differ significantly (i.e., a large residual exists), the signal may be reporting abnormal behavior in the monitored asset.

[0044] In one embodiment, an ML model is trained to generate an estimate of a predicted observation based on training observations that represent normal operation. The ML model is trained or configured to generate an estimate that is a predicted observation based on observations in a cyclic double buffer. A supervised learning process is performed on the machine learning model and the training data to train the machine learning model. The training data may be a set or series of observations in which, given other signal values ​​in the observations, the signal observations in each observation are defined as normal measurements, nominal measurements, or standard measurements. The observations in the training set represent the normal operation of the monitored asset, where the monitored asset is functioning within appropriate or expected parameters. The observations in the training set may be live data streamed from a stream of observations. The observations in the training set may also be from a historical database of time-series signals.

[0045] In a supervised learning process, the parameters of the ML model's function are iteratively adjusted with the aim of generating estimates that approximate the observations in the training data. The supervised learning process terminates when the ML model consistently generates estimates that are sufficiently similar to the observations in the training data. The trained ML model may then be used to generate estimates from observations received from a stream of observations.

[0046] In one embodiment, the machine learning model is implemented as one or more non-linear non-parametric (NLNP) regression algorithms used for multivariate pattern recognition or anomaly detection. These ML anomaly detection algorithms may include similarity-based modeling (SBM), such as neural networks (e.g., long short-term memory (LSTM) networks), support vector machines (SVMs), auto-associative kernel regression (AAKR), and multivariate state estimation techniques (MSET). Thus, in one embodiment, the ML model is an NLNP model or an MSET model. Also, in one embodiment, the ML model is a multivariate machine learning model that operates on a set of multiple variables as input and generates estimates of multiple variables.

[0047] In one embodiment, the ML model generates an estimated value of the expected observation from the observations in the first buffer while the second buffer is being read from a real-time stream of observations. The ML model also generates an estimated value of the expected observation from the observations in the second buffer while the first buffer is being read from a real-time stream of observations. In one embodiment, the reading or retrieval of estimates from alternating sources (first and second buffers) in a cyclical dual buffer is repeated in a repeating cycle in response to the continuous arrival of new observations from the stream of observations. Thus, the streaming ML estimation method 200 generates estimates using the ML model from observations in one of the two buffers simultaneously while the other buffer is being read from the stream of observations.

[0048] With respect to two separate buffers (a first buffer and a second buffer) that constitute a circular dual buffer, read operations that retrieve observations from the buffers for processing into the ML model are moved to the buffer when the buffer is full. In one embodiment, an outflow pointer indicates which of the two buffers a batch of observations is being read from for processing. The outflow buffer indicates which buffer is the source from which the ML model is drawing observations to generate estimates. The outflow pointer alternates between pointing to the first buffer and pointing to the second buffer. The outflow pointer points to the opposite of the intake pointer, indicating the source buffer. In one embodiment, the outflow pointer is a variable that stores the address of one of the two buffers in memory, or contains such a variable. In one embodiment, the outflow pointer switches buffers in response to the same events as the intake pointer (described with reference to process block 215 above). The outflow pointer may be updated to point to the first buffer (i.e., to store the address of the first buffer in memory) in response to the last position of the first buffer being filled, and the outflow pointer may be updated to point to the second buffer in response to the last position of the second buffer being filled.

[0049] In one embodiment, the streaming ML estimation method 200 generates estimates from observations in two buffers (the first and second buffers of a circular double buffer) by a machine learning model run by a primary compute instance. The streaming ML estimation method 200 also uses an auxiliary compute instance to load the observations into the two buffers. The primary and auxiliary compute instances may be, for example, separate virtual machine (VM) instances or separate cloud containers. The two compute instances share access to memory and / or storage. The circular double buffer is stored in memory accessible to both compute instances.

[0050] In one embodiment, both the primary and auxiliary computing instances operate a machine learning application. The machine learning application includes functions for running a machine learning model to generate estimates, functions for reading observations from a stream of observations into a cyclic double buffer, and functions for writing the estimates generated by the ML model to an estimates output stream. In one embodiment, the primary computing instance executes the functions of the machine learning application that cause the machine learning model to generate estimates. In one embodiment, the auxiliary computing instance executes the functions of the machine learning application that cause observations to be read into a cyclic double buffer, and functions of the machine learning application that cause estimates (as well as related data such as observations and residuals corresponding to the estimates) to be written to an output stream. In one embodiment, the auxiliary computing instance may also be referred to herein as an input / output handler.

[0051] In one embodiment, the streaming ML estimation method 200 generates a first estimate (a first vector of estimates) from observations (vectors of observations) in a first buffer (of a cyclic double buffer) using a multivariate machine learning model. The streaming ML estimation method 200 also generates a second estimate (a second vector of estimates) from observations (vectors of observations) in a second buffer (of a cyclic double buffer) using a multivariate machine learning model. In one embodiment, an auxiliary computation instance generates batches of observations for processing by loading observations from the stream into the first and second buffers while the primary computation instance is simultaneously and in parallel generating ML estimates from previously loaded batches. In one embodiment, the previously loaded batch is loaded into one buffer immediately before the loading of other buffers currently being loaded.

[0052] In one embodiment, the streaming ML estimation method 200 switches which of the two buffers in a circular dual buffer is being read and which is being processed in response to the buffer currently being read becoming full. For example, in parallel with the generation of a first estimate by the machine learning model, the streaming ML estimation method 200 places observations from the stream into the second buffer until the second buffer is filled. In response to the second buffer being filled, the streaming ML estimation method 200 generates a second estimate from the observations in the second buffer using the machine learning model. In parallel with the generation of the second estimate by the machine learning model, the streaming ML estimation method 200 places observations from the stream into the first buffer until the first buffer is filled. In response to the first buffer being filled, the streaming ML estimation method 200 generates a first estimate from the observations in the first buffer using the machine learning model.

[0053] In one embodiment, this cycle repeats indefinitely as long as no estimates of new observations received from the stream of observations are generated. The ML model cycles or alternates in taking observations for estimation from the first and second buffers in a cyclic double buffer. This is between taking a first batch of observations from the first buffer while the second buffer is being read, and taking a second batch of observations from the second buffer while the first buffer is being read. Thus, while the currently filled buffer is not full, the streaming ML estimation method 200 uses the machine learning model to generate a first vector of estimates from the vector of observations in the currently unfilled buffer.

[0054] In one embodiment, using an auxiliary computation instance to fill the currently filled buffer and output the estimate is performed in parallel with using a primary computation instance to generate the estimate from a vector of observations in the currently unfilled buffer. In one embodiment, a circular double buffer thus allows the input / output operation to be performed in parallel with the generation of the estimate. In this way, processing delay time is reduced.

[0055] Therefore, in one embodiment, the streaming ML estimation method 200 generates an estimate from observation data from one buffer (of the circular buffer) that was previously read in parallel (simultaneously) while observations from the stream of observations are being read into another buffer (of the circular buffer), and then alternates between the two buffers based on which buffer is currently being read. After process block 220 is completed, the streaming ML estimation method 200 continues in process block 225. At the end of process block 220, the ML estimates of the observations should have been generated for the previously read batch of observations in parallel while another batch of observations is being read. The ML estimates may be written to the estimate output stream.

[0056] -Exemplary Streaming ML Estimation Method-Writing Estimates to a Stream- In process block 225, the streaming ML estimation method 200 writes the estimates to an estimate stream in real time as the estimates are generated. In one embodiment, the estimates generated by the ML model are streamed externally as they are generated.

[0057] As described above, in one embodiment, the estimate is a data structure that includes a vector of estimates for a set of signals, and a timestamp or other index. In one embodiment, the estimate may also include other data in the data structure. In one embodiment, the estimate may include both a vector of estimates for a set of signals and a vector of observations corresponding to the set of signals, thus including both observations and estimates together. In one embodiment, the estimate may include both a vector of estimates for a set of signals and a vector of residuals corresponding to the set of signals, thus including both estimates and residuals from observations together. In one embodiment, the estimate may include a vector of estimates for a set of signals, a vector of observations corresponding to the set of signals, and a vector of residuals corresponding to the set of signals. The "corresponding" estimates, observations, and residuals coincide with respect to timestamps or other indexes.

[0058] In one embodiment, the stream of estimates is defined by a streaming function that streams data values. The streaming function monitors a specific location in memory used by the ML model to output estimates. When generated by the streaming ML estimation method 200, the estimate data structure generated in process block 220 is placed in the location in memory monitored by the streaming function. The streaming function detects the writing of estimates to the location in memory. In response to detecting the writing of estimates to the location in memory, the streaming function retrieves the estimates and then sends them to a destination. This destination may include, for example, a location in memory or storage, another program or function, or a network location. For example, the destination may be an anomaly detection test program or function for determining whether the residual between the observed and estimated values ​​is anomaly (for example, as described below).

[0059] In one embodiment, estimates are written to the stream one estimate at a time. Estimates are written in real time. In one embodiment, real-time writing of estimates to the stream does not wait for one or more subsequent estimates to be generated before starting to write estimates to the stream. Instead, for example, estimates are written to the output stream immediately upon generation of an estimate, in response to the generation of the estimate.

[0060] In one embodiment, estimates are written to a stream by an auxiliary computation instance. In one embodiment, estimates are written to a stream by an auxiliary computation instance in parallel with ML estimate generation by the primary computation instance. For example, the streaming ML estimation method 200 uses an auxiliary computation instance to write estimates to an estimate output stream. In one embodiment, the auxiliary computation instance writes generated estimates while the primary computation instance is generating subsequent estimates from observations stored in a cyclic double buffer. The auxiliary computation instance writes estimates being generated by the primary computation instance while the primary computation instance is simultaneously generating subsequent estimates. Since the writing of estimates to the stream is processed in parallel with the subsequent generation of estimates, the delay time for writing output to the stream is masked by the computation delay time for estimate generation. This further reduces the end-to-end delay time.

[0061] In one embodiment, the streaming ML estimation method 200 monitors the output of an ML model for the generation of estimates, and then, upon detection of the generation of estimates by the ML model, the estimates are quickly retrieved and sent to a destination (such as a downstream anomaly detection test function or program) while the ML model is operating to generate subsequent estimates. After that, process block 225 is completed, and the streaming ML estimation method 200 proceeds to termination block 230, in which termination block 230 completes the method 200.

[0062] Upon completion of the streaming ML estimation method 200, the ML estimates of the observations received from the stream are generated in real time and streamed externally in a manner that reduces the end-to-end latency required for the streaming ML estimation process. With respect to the use of a circular double buffer, the steps described herein enable the streaming ML estimation method 200 to generate estimates in a stream with only the minimum latency consumed by the input and output operations, the majority of which is attributable to the actual generation of the ML estimates. In one embodiment, this allows for earlier detection and warning of anomalies than other methods for streaming estimation generation.

[0063] -Further Embodiments of Streaming ML Estimation Method- In one embodiment, before initiating parallel processing of ML estimation and input / output activities (using the aforementioned circular double buffer arrangement of the first and second buffers), this method determines whether sequential processing of input, ML estimation, and output is too slow. Thus, in one embodiment, before reading observations into the circular buffer in real time, the end-to-end delay time of sequential processing of observations to estimates is measured, and if the end-to-end delay time of sequential processing is too long, the processing is switched to parallel processing.

[0064] In one embodiment, before reading observations into a circular buffer in real time, the streaming ML estimation method 200 sequentially reads observations, generates estimates from the observations, and further measures the end-to-end delay time for writing the estimates to a stream of estimates. The streaming ML estimation method 200 then compares the end-to-end delay time to a threshold of maximum allowable delay time. The streaming ML estimation method 200 then switches to reading observations into a circular buffer in response to the threshold being met.

[0065] If the threshold is met (i.e., the end-to-end latency exceeds the threshold), sequential processing is too slow (end-to-end latency is too long), and parallelized processing should take over to ensure sufficient throughput to keep pace with the stream of observations. If the threshold is not met (i.e., the end-to-end latency is below the threshold), sequential processing is fast enough (end-to-end latency is too short), allowing for sufficient throughput. In this case, the streaming ML estimation method 200 continues to process observations sequentially. The streaming ML estimation method 200 may recheck from time to time to determine whether the threshold being met indicates a transition to parallelized processing.

[0066] In one embodiment, the operation of the streaming ML estimation method 200 is parallelized across two computation instances: a primary computation instance for generating ML estimates and an auxiliary computation instance for handling streaming input and streaming output operations. In one embodiment, the primary and auxiliary computation instances are virtual machines. In one embodiment, the primary and auxiliary computation instances are instances of a containerized machine learning application. Both the primary and auxiliary instances have access to first and second buffers.

[0067] Therefore, in one embodiment, the streaming ML estimation method 200 further instantiates a primary computation instance. The primary computation instance is configured to generate estimates by the ML model. The streaming ML estimation method 200 also further instantiates an auxiliary computation instance. The auxiliary computation instance is configured to read observations into a circular buffer and write estimates to a stream of estimates. The streaming ML estimation method 200 places the circular buffer (containing the first and second buffers) in a location in memory accessible from both the primary and auxiliary computation instances. Further details regarding the computation instances are described above with reference to process block 220 and below with reference to, for example, Figure 3.

[0068] Estimates at an index point (such as a timestamp) are generated from observations at the index point. In this way, estimates and observations correspond when the index point of an estimate is the index point of an observation. In one embodiment, an estimate is accompanied by the corresponding observation from which the estimate was generated when it is written to the stream of estimates. For example, an ML model may generate an estimate as a data structure containing both the estimate and observation for a single timestamp. If the observation is a vector containing per-signal observations in a set of signals at a timestamp, the estimate may be a vector containing both per-signal observations in a set of signals at a timestamp and per-signal estimates in a set of signals. Thus, in one embodiment, the streaming ML estimation method 200 further writes the observations corresponding to the estimate, along with the estimate, to the stream of estimates.

[0069] The residual or difference between an observed value and an estimated value may be used to detect anomalies in the observed value. In one embodiment, the ML model may generate the residual by subtracting the estimated value from the corresponding observed value. The estimated value, observed value, and residual value correspond when the index points of the residual are the index points of the observed value and the estimated value. In one embodiment, when an estimated value is written to a stream of estimates, it is accompanied by the estimated value and the corresponding residual between the estimated value and the corresponding observed value from which the estimated value was generated. For example, the ML model may generate an estimated value as a data structure that includes both the estimated value and the residual for a single timestamp. If the estimated value is a vector containing the estimated value for each signal in a set of signals at a timestamp, the estimated value may also include the residual value between the estimated value and the observed value for each signal in a set of signals at a timestamp. Thus, in one embodiment, the streaming ML estimation method 200 further writes the residual corresponding to the estimated value, along with the estimated value, to the stream of estimates.

[0070] As described above, referring to process blocks 210 and 220, in one embodiment, the observations and estimates are vectors containing the values ​​of multiple signals. Also in one embodiment, the machine learning model is a multivariate ML model that generates estimates for each signal based on observations of signals other than the signal for which estimates are being generated. Thus, in one embodiment, the observations are a vector containing the observations for each signal in the set of signals. Also, the estimates are a vector containing the estimates for each signal in the set of signals. Also, the machine learning model is a multivariate machine learning model. Next, generating estimates (as described in process block 220) further includes generating estimates for this one signal in the set of signals based on observations of the other signals in the set. Thus, estimates may be generated for each signal in the set from observations of the other signals in the set.

[0071] As described herein, ML models are used to detect anomalies in a stream of observations based on residuals between corresponding observations and estimates. An observation in a signal is anomaly if it deviates from an estimate in a way that satisfies a threshold test for anomaly detection. In one embodiment, when an anomaly is detected, a warning indicating the presence of an anomaly may be generated. In one embodiment, the streaming ML estimation method 200 further generates a warning that an anomaly has been detected in response to one or more residuals between one of the observations and one of the estimates of that observation satisfying a threshold.

[0072] In one embodiment, to generate a warning, the streaming ML estimation method 200 analyzes the residuals between the observed and estimated values ​​by an anomaly detection test to determine whether the observed value is anomaly. In one embodiment, the residuals may be analyzed from the estimated value data structure if they are included in the estimated value data structure as described above, or they may be calculated from the estimated and observed values. In one embodiment, the anomaly detection test is a sequential probability ratio test (SPRT). The SPRT calculates a cumulative sum of log-likelihood ratios for each consecutive residual between the observed value and the estimated value of the signal, and compares this cumulative sum to a threshold indicating an anomaly. If the threshold is exceeded, an anomaly is detected.

[0073] In one embodiment, if one or more observations in the observation are detected as abnormal, a warning may be generated to notify the user or other system of the presence of an anomaly. In one embodiment, the warning is an electronic message. The warning may be generated by creating an electronic message and then presented by transmitting the generated warning for subsequent display or other action. The warning may be configured to be displayed in a graphical user interface. For example, the warning may be used to notify the operator of a monitored asset that the asset is behaving abnormally or in an unexpected manner. The warning may be configured as a request (such as a REST request) used to trigger the initiation of some other function. For example, the warning may be used to initiate automatic adjustment or shutdown of the monitored asset's operation, or to initiate an automated maintenance request regarding the monitored asset. The warning may be generated and presented in real time after the detection of an anomaly, enabling a rapid response to the detected anomaly.

[0074] In one embodiment, the streaming ML estimation method 200 determines the buffer length, for example, by receiving a buffer length input from a user, or by experimentally or automatically determining the buffer length. In one embodiment, a batch size, which is the length of the first and second buffers, may be provided to the streaming ML estimation method 200, and the buffer lengths are adjusted to accommodate the batch size. The batch size (or buffer length) may be a value input by a user. In one embodiment, the batch size is measured in terms of the number of observation vectors or rows. In one embodiment, the batch size is the number of observation vectors stored in one buffer of the cyclic double buffer before it becomes full, switches to processing the buffer, and stores the vectors in the other buffer of the cyclic double buffer. In one embodiment, a user or administrator inputs the buffer length to the system, for example, through a user interface, in order to configure the streaming ML estimation system 100. Once the buffer length is provided, the streaming ML estimation method 200 configures the first and second buffers to have a length that is the provided buffer length, i.e., the number of available rows. Therefore, in one embodiment, the streaming ML estimation method 200 further receives a buffer length input and configures the first and second buffers to receive the same number of observations as the buffer length before they become full.

[0075] In one embodiment, the batch size or buffer length may be automatically determined experimentally for a particular use case. Where used herein, the term “use case” refers to a specific configuration of streaming speed and signal count (number of observations per observation). In one embodiment, the buffer length is automatically identified, causing the streaming ML estimation method 200 to maintain the same pace as the stream of observations when generating ML estimates for a given number of signals. Thus, in one embodiment, the streaming ML estimation method 200 further automatically identifies the buffer length and configures the first and second buffers to receive a number of observations equal to the buffer length before they become full.

[0076] In one embodiment, in order to automatically identify the buffer length (batch size) for use cases, the batch-by-batch processing delay time for performing the streaming ML estimation method 200 on a set of test data is repeatedly measured for different batch sizes. The test data includes the number of signals for use cases. Observations from the test data are streamed to the streaming ML estimation system 100 at the streaming rate for use cases. If the total processing delay time for a batch is shorter than the time required to stream all observations in the batch, processing can maintain the same pace as the stream in real time. Therefore, in one embodiment, the processor identifies a batch size in which the measured batch-by-batch processing delay time is shorter than the time required to stream all observations in the batch (streaming rate × batch size). In one embodiment, the measured batch-by-batch processing delay time should be shorter by a margin, such as 20%, than the time required to stream all observations in the batch, in order to account for possible variations in processing delay time.

[0077] In one embodiment, one of the identified batch sizes is automatically selected to be the buffer length. In one embodiment, the processor automatically selects the identified batch size that minimizes the batch-by-batch processing delay. Note that, as described below, the processing delay is dominated by read and write delays for smaller batch sizes and by ML estimation delays for larger batch sizes. If the minimum batch-by-batch processing delay for a use case remains greater than the time required to stream all observations in a batch, a faster computational configuration may be shown. In one embodiment, the above process for automatic identification of batch size (buffer length) may be repeated incrementally for faster computational configurations until it is determined that the computational configuration and buffer length can keep pace with streaming. Further details regarding batch size / buffer length identification are described below in this specification.

[0078] In one embodiment, the ML model is trained before being used to generate estimates. In one embodiment, the ML model is trained to generate estimates of expected observations based on training observations that represent normal operation. Further details regarding the training of the ML model are described above with reference to process block 220.

[0079] In one embodiment, the ML model is a Multivariate State Estimation Technique (MSET) model, as described above in process block 215.

[0080] In one embodiment, one or more non-temporary computer-readable media include stored computer-executable instructions (also called program instructions). The computer-executable instructions are configured to cause one or more computers to perform an operation, including the operation of the streaming ML estimation method 200 (or other methods described herein) when executing the computer-executable instructions.

[0081] In one embodiment, the computing system includes one or more computers. The computing system is configured to perform operations by computer executable instructions, including the operation of the streaming ML estimation method 200 (or other methods described herein).

[0082] In one embodiment, the computer program product includes a computer program, which, when executed by at least the computer's processor, causes the computer to perform an operation including the operation of the streaming ML estimation method 200 (or other methods described herein) by computer-executable instructions. For example, the computer program may include one or more computer-executable instructions that cause the computer to perform an operation.

[0083] - Additional Contexts and Embodiments - Multivariate machine learning (ML) can be used to monitor time-series signals for automated anomaly detection. Typically, ML anomaly detection services analyze data in static batches, for example, from a previously recorded data historian database. ML anomaly detection can be extended to real-time streaming mode, allowing new time-series streaming from live sensors to be analyzed in real-time mode to detect anomalies (and trigger alarms) as quickly as possible. In one example, ML anomaly detection can be extended to real-time streaming mode by performing loop batch processing (e.g., by iterative calls to ML functions) to sequentially analyze batches (e.g., frames or buffers) of customer data.

[0084] One challenge in converting static batch-based ML analysis is pre-estimating the throughput and latency of large-scale data ingestion and processing in streaming mode. In one embodiment, streaming mode should be configured to keep up in real time with respect to a specific number of signals and a specific sampling rate selected for the data being monitored. Otherwise, the system will crash and / or latency will increase indefinitely when the amount of data throughput (i.e., the number of signals at the sampling rate) reaches a level that processing cannot keep up in real time. Both situations can lead to timeout exceptions at the upstream data ingestion hardware / software interface, or even cause data to be discarded at the data ingestion interface.

[0085] In the analysis of ML anomaly detection algorithms, the challenge of estimating end-to-end real-time processing delays is complicated by the fact that ML computation costs increase non-linearly with the number of signals being monitored, but linearly with the product of the sampling rate and buffer size of the data being analyzed. Furthermore, there are read delay components in the processing delay for filling the buffer, and write delay components for writing the results to the output stream.

[0086] For example, compare a simple throughput architecture (or "pipe") for ML processing, a simple throughput architecture (or "pipe") with typically directly proportional processing time, and a simple throughput architecture (or "pipe") with no processing. In the throughput architecture for ML processing, the incoming data stream fills a buffer for processing, which is then processed by the ML algorithm when full, and the results are written to the output stream. In the simple throughput architecture with no processing, no ML computation is performed when the buffer is full; the data is simply moved from input to output.

[0087] In an exemplary, simple throughput architecture with no processing, estimating end-to-end latency is trivial, with read latency for filling the buffer and write latency for outputting the buffer. When moving data in large batches through a pipe (e.g., with a buffer length of 100,000 rows), read and write latency are small. However, when moving data in small batches through a pipe (e.g., with a buffer length of 10 rows), the read and write latency components can dominate the overall latency for sustained throughput of the streaming time series. Nevertheless, for non-ML applications (where batch calculations are not performed on the buffer contents), estimating the overall streaming latency is straightforward. The cumulative latency is a simple linear function of the buffer length (or the number of rows per chunk of data).

[0088] In a typical, simple throughput architecture for ML processing, end-to-end latency decomposition becomes significantly more difficult when ML computations are performed on the contents of a buffer. For ML anomaly detection algorithms used on time series, the computational cost increases quadratically with the number of signals in the buffer and linearly with the number of rows in the buffer. This raises complex, non-linear trade-offs between the number of signals, the number of rows, the buffer length, and the computation shape (e.g., various types of central processing units (CPUs) and graphics processing units (GPUs)), each of which can significantly alter the processing latency, or the time the ML spends processing the contents of the buffer.

[0089] Therefore, for ML anomaly detection techniques (such as Multivariate State Estimation Techniques (MSET), neural networks, and Support Vector Machines (SVMs)), the challenge of estimating end-to-end real-time processing latency is complicated by the fact that ML computation costs increase quadratically with the number of signals, but linearly with the product of the sampling rate and buffer size of the data being analyzed. There are still read latency components for filling the buffer and write latency for writing the results to the output stream. However, estimating end-to-end throughput rate and cumulative latency (to ensure that streaming computations keep up with real-time ingestion of time-series data) has become significantly more difficult. Furthermore, determining the latency of end-to-end throughput for streaming ML anomaly detection techniques is not well-suited to brute-force trial and error, such as by incrementing the number of signals and / or sampling rate until the overall throughput can no longer keep up in real time. This is simply because changes to the number of signals or sampling rate for new use cases change the bottleneck where the streaming / processing architecture can no longer keep up with the incoming ingestion rate.

[0090] Processing time (or computational cost) for streaming data processing that is not associated with input and output can range from minimal, nearly zero, when simply passing data over an I / O channel, to processing time that is directly proportional to the amount of data (and therefore easily measurable and scalable), and further to the complex, non-linear growth of ML anomaly detection processing. Examples of minimal, near-zero processing costs include pass-through streaming of video and archiving data from a streaming source to a storage system. Examples of linearly scalable processing costs include taking the square root of every number passing through a pipe, encrypting / decrypting data, converting units in time series, statistically scaling data, and language conversion. Conventional techniques for estimating end-to-end latency are straightforward for simple systems with no processing cost for "data in transit," and also for use cases where processing cost scales linearly with the "volume throughput" of data (in which case it is straightforward to measure the processing overhead of small streams of data and scale linearly up to the architecture's bandwidth capacity).

[0091] Estimating end-to-end throughput and latency for time-series data streams using ML anomaly detection processing, and especially for multivariate pattern recognition, is challenging. Examples of multivariate tools / technologies include multivariate state estimation techniques (MSET) (including Oracle's proprietary MSET2), neural networks (including long short-term memory (LSTM) networks), and support vector machines (SVMs).

[0092] The challenge introduced by these multivariate anomaly detection techniques is that computational cost (and therefore processing delay) changes non-linearly with the amount of time-series data. For example, the computational cost of a particular buffer of data scales quadratically with the number of signals in the buffer and linearly with the number of rows in the buffer. Thus, for example, the processing time of a buffer of time-series data containing N columns of signals (from sensors) and M rows or M vectors (timestamps) will be significantly different from the processing time of a buffer of the same size containing M columns and N rows (assuming M is not equal to N).

[0093] It should be noted that the signals in anomaly detection use cases may include a mixture of univariate and multivariate (meaning correlated) signals. The preprocessing step for anomaly detection identifies and separates the univariate signals from the multivariate signals for individual processing. This preprocessing analysis is a one-time upstream offline analysis and is therefore not part of estimating the throughput and latency of streaming multivariate ML anomaly detection.

[0094] In this specification, two processing modes for multivariate ML anomaly detection are introduced in one embodiment. The first processing mode uses a single circular buffer for multivariate ML anomaly detection on a streaming signal. The second processing mode uses a circular dual buffer for multivariate ML anomaly detection on a streaming signal. Each method may be used, for example, in a cloud implementation, cloud edge implementation, or local server implementation of multivariate ML anomaly detection on a streaming signal.

[0095] In general, in multivariate ML time series analysis, calculations operate on a data buffer. In one embodiment, the data buffer is an array of time series values. In the array of time series values, columns represent sensors and rows represent timestamps or observations (however, the reverse rule may also be used, where columns represent timestamps and rows represent sensors). One example may be signals from an asset containing 30 sensors, where 1000 timestamped observations are read at a time into a buffer with 30 columns and 1000 rows. Continuous buffering of streaming data is typically used for processing streaming data.

[0096] In one embodiment, the buffer is a circular buffer. The circular buffer is configured to temporarily store streaming data. In a circular buffer, the buffer is configured to be filled with new incoming data and emptied when the results are written out after the completion of the ML anomaly detection analysis. The buffer length (number of rows) remains the same for each application of the ML algorithm. In the simple example above using 1000 rows, the circular buffer is filled with an incoming stream of 1000 rows of data that are ingested, the 1000 rows of data are analyzed by the ML algorithm to identify residuals between observations and estimates, and then 1000 rows of results (corresponding to the 1000 rows of data) are written to the output stream.

[0097] In single-cycle buffer streaming throughput processing, the processes of reading, ML analysis, and writing occur in a linear order for each batch of data. At the start of operation, there is a "read delay," which is the time required to fill the buffer. The read delay increases linearly with the increase in the amount of data in the buffer (number of signals multiplied by the sampling rate). There is an ML analysis delay, which is the time required to perform ML analysis on the contents of the buffer. The ML analysis delay scales non-linearly with the amount of data in the buffer. More specifically, the ML analysis delay scales quadratically with the number of signals (columns) but linearly with the number of observed (row) vectors. There is a write delay, which is the time required to write the results to the output stream. Similar to the read delay, the write delay is also linear with the amount of data in the buffer. The write delay may be longer than the read delay because the output result may contain more columns than the input stream. For example, ML analysis outputs the calculated results of those signals in addition to the original raw time-series signals, meaning the output stream has twice the number of columns as the input stream. However, the output write delay time is still linearly proportional to the amount of data in the buffer.

[0098] Note that in certain use cases (where the number of sensors is fixed and cannot be changed), there may be significantly different overall throughput latency depending on the selected buffer length (i.e., number of rows per buffer) when using a single-buffer configuration. If a single circular buffer exists, depending on the length (in rows) of the circular buffer selected for the batch processed by ML, decomposing the latency involves three key components: read latency (proportional to the number of input rows multiplied by the number of signals), processing (ML analysis) latency (scales with the number of input rows multiplied by the square of the number of signals), and write latency (proportional to the number of output rows multiplied by the number of signals). For any specific use case, a wide range of end-to-end latency may exist depending on the length of the circular buffer in a non-trivial way. The length of the circular buffer may be user-configurable. End-to-end latency can be parametrically evaluated precisely as a function of window width (number of rows) in a single-buffer framework for ML anomaly detection in streaming data.

[0099] Real-time ML analysis of large sensor groups can challenge or exceed the capabilities of available computer processing specifications. It should be noted that, at the time of writing, individual commercial aircraft may have more than 75,000 sensors, and modern oil refineries or medium-sized data centers may each have more than 1,000,000 sensors. The circular double buffer, an alternating pointer framework for ML anomaly detection in streaming data, is presented herein as an alternative to the use of single-buffer frameworks in use cases where end-to-end throughput latency exceeds the maximum latency requirements of large-scale real-time processing applications. The circular double buffer technique hides all (or nearly all) of the read / write latency, as will be described in more detail elsewhere in this specification.

[0100] In a standard streaming application using the single - loop buffer method, the latency is characterized by three latency components per buffer of data: read time, processing (ML analysis) time, and write time. Therefore, in sequential single - buffer processing, the end - to - end total latency L T after B buffers is the sum of the read latency L R , the processing latency L P , and the write latency L W multiplied by B (L T = B×(L R +L P +L W )). The total latency is linearly proportional to the buffer size or the dimensions of the buffer in terms of the measured values (rows) and the number of signals. However, as the number of signals in the buffer processed by the ML model increases, the processing latency L P increases quadratically.

[0101] Note that the three latency components are typically applied to ML anomaly - detection techniques including MSET, neural networks, and SVM. Hyperparameters specific to the ML technology (such as the numvecs (number of vectors) hyperparameter in MSET2) are not related to the latency. Thus, in one embodiment, the latency calculations and circular double - buffer streaming shown and described herein are compatible with any ML technology.

[0102] As an alternative to the single-buffer method, where read and write delays accumulate cumulatively with the processing delay for ML anomaly detection, the circular double-buffer method hides read and write delays, thereby maximizing the contents of each buffer. In one embodiment, the circular double-buffer method starts two computation instances, each computation instance having its own dedicated buffer. Each of the two dedicated buffers is accessible by both computation instances. The two dedicated buffers may be connected end to end in a circular configuration to form a circular double buffer.

[0103] In one embodiment, following a circular double buffering technique, both the intake and outflow pointers alternately repeat two dedicated buffers. The intake pointer indicates, for example, by an auxiliary instance, which of the two dedicated buffers constituting the circular double buffer is currently reading the incoming observation. The outflow pointer indicates which of the two dedicated buffers is currently being read by the ML model. The intake and outflow pointers indicate opposing buffers, which are the two dedicated buffers constituting the circular double buffer. While the primary instance is performing the computation, the auxiliary instance is reading the next batch of data.

[0104] In this method, for the same batch of B data as in the previous example, one read delay time L R and B processing delay time L P and one write delay time L W The delay time (L) is the sum of the above. T =L R +(B×L P )+L WOnly exists. Therefore, the B occurrences of the combined read and write delays are removed from the end-to-end total delay. Thus, the cyclic double buffer, which is an alternating pointer technique, hides the delays of the read and write operations by performing the read and write operations in parallel with auxiliary computation instances. Further details regarding the operation of the cyclic double buffer technique using alternating pointers are shown herein with reference to Figure 3.

[0105] Figure 3 shows range chart 300 for single-buffer operation and range chart 305 for circular double-buffer operation using alternating pointers to switch buffers. Range charts 300 and 305 are decomposed into components and show the delay times allocated between computation instances. Range chart 300 for single-buffer operation shows the sequence in which read, processing, and write operations are performed in a single-buffer configuration using one computation instance 310. Range chart 305 for circular double-buffer operation shows the sequence in which read, processing, and write operations are performed in a circular double-buffer configuration using a primary computation instance 315 and an auxiliary computation instance 320. Read, processing, and write operations are shown along the computation time axis 325. A read block (shown with diagonal hashing) requires approximately a first amount of computation time to complete. A processing block (shown without hashing) requires approximately a second amount of computation time to complete. A write block (shown with cross-hatching) requires approximately a third amount of computation time to complete.

[0106] In a single-buffer configuration using one compute instance 310, the compute instance 310 reads the first batch of data in the read block 330, adding the computation time for the first amount to the end-to-end delay time. Next, the compute instance 310 processes the first batch of data in the process block 331 to generate the ML estimate for the first batch, adding the computation time for the second amount to the end-to-end delay time. Next, the compute instance 310 writes the ML estimate for the first batch of data to the output stream in the write block 332, adding the computation time for the third amount to the end-to-end delay time. This process is then repeated for subsequent batches of data. For example, the compute instance 310 reads the second batch in the read block 333, adding the computation time for the first amount to the end-to-end delay time again. The compute instance 310 processes the ML estimate for the second batch in the process block 334, adding the computation time for the second amount to the end-to-end delay time again. Furthermore, in write block 335, one computation instance 310 writes the estimate to the stream, adding the computation time of a third quantity to the end-to-end delay time again.

[0107] In a circular double-buffer configuration using a primary compute instance 315 and an auxiliary compute instance 320, the auxiliary compute instance 320 reads the first batch of data in read block 340 and adds the computation time of the first amount to the end-to-end delay time. Next, the primary compute instance 315 processes the first batch of data in processing block 341 to generate ML estimates for the first batch and adds the computation time of the second amount to the end-to-end delay time. While the primary compute instance 315 is generating ML estimates in processing block 341, the auxiliary compute instance 320 is reading the second batch of data in read block 342. Read block 342 consumes the computation time of the first amount, but the computation time of the first amount in read block 342 is not added to the end-to-end delay time because read block 342 is executed simultaneously by the auxiliary compute instance 320 while the primary compute instance 315 is executing processing block 341.

[0108] The second batch is read by the auxiliary computing instance 320 by the time the primary computing instance 315 has finished generating estimates for the first batch. The primary computing instance 315 then processes the second batch of data in processing block 343 to generate ML estimates and adds the computation time for the second batch to the end-to-end delay time. While the primary computing instance 315 is generating ML estimates for the second batch in processing block 343, the auxiliary computing instance 320 is (i) writing estimates for the first batch of data to the output stream in write block 344 and (ii) reading the third batch of data in read block 345. The read block 345 consumes a first amount of computation time, and the write block 344 consumes a third amount of computation time. However, while the primary compute instance 315 generates an estimate for the second batch in block 343, the read and write blocks are performed by the auxiliary compute instance 320, so the first and third amounts of computation time for the read block 345 and write block 344 are not added to the end-to-end delay time.

[0109] In one embodiment, the read and write operations performed by the auxiliary compute instance 320 are shown as separate blocks in Chart 305 of the circular double buffer operation, but may be mixed together. In one embodiment, the batch write operation is shown in Chart 305 of the circular double buffer operation as occurring after the batch processing, but is not delayed until after the batch of ML estimates has been processed. For example, as described above, individual estimates are written to the estimates stream by the auxiliary compute instance 320 immediately upon generation by the primary compute instance 315. Thus, some individual estimates from a batch may be written to the estimates stream simultaneously with the generation of other estimates in the batch.

[0110] This process continues in a similar manner through subsequent batches of data. For example, while the primary computing instance 315 is generating the ML estimate for the current batch of data, the auxiliary computing instance 320 reads subsequent batches of data and writes the estimates. This operation adds only a second amount of computation time per batch to the end-to-end delay time until after the last batch. After the estimate for the last batch of data has been generated, the auxiliary computing instance 320 performs the final write operation for the last batch, adding a third amount of computation time to the end-to-end delay time.

[0111] Figure 4 shows an exemplary graph 400 of the total delay time 405 against the number of signals for streaming ML anomaly detection using a single-buffer processing configuration. The total delay time 405 against the number of signals is plotted against a first axis 410 representing the number of signals in the buffer and a second axis 415 representing the total delay time for streaming ML anomaly detection. Note that the line for the total delay time 405 against the number of signals is not smooth due to the somewhat random amounts of time required by read, write, and read operations, which accumulate along with the time required for ML anomaly detection.

[0112] To demonstrate the improved latency after the adoption of the circular double-buffering technique, the streaming use case presented in Figure 4 for the single-buffering technique is re-executed using the circular double-buffering technique, as shown in Figure 5. Note that when measuring latency, only the total end-to-end latency that occurred is measured, as the objective is to minimize the total end-to-end latency.

[0113] Figure 5 shows an exemplary graph 500 of the total delay time 505 against the number of signals for streaming ML anomaly detection using a cyclic double buffer processing configuration. The total delay time 505 against the number of signals is plotted against a first axis 510 representing the number of signals in the buffer and a second axis 515 representing the total delay time for streaming ML anomaly detection. As can be seen by comparing Figure 4 and Figure 5, the total delay time is orders of magnitude smaller. For example, the total delay time is reduced by approximately 90%. Also, the line for the total delay time 505 against the number of signals using the cyclic double buffer processing configuration is smoother than the line for the total delay time 405 against the number of signals using a single buffer configuration. The line for the total delay time 505 against the number of signals is smoother because random time read, write, and read operations are not performed in series with the ML anomaly detection process. For example, in one embodiment, read, write, and read operations are not performed by the first computation instance that also performs the ML anomaly detection process. Instead, the load of these prominent buffers is hidden. For example, in one embodiment, the read, write, and read operations are instead performed by a second computation instance that does not perform ML anomaly detection processing.

[0114] As described herein, transitioning from a processing loop using a single circular buffer to a processing loop using a circular double buffer significantly reduces end-to-end latency for anomaly detection in streaming ML, but typically no further reduction in latency can be achieved by adding more buffers. This is because, as previously mentioned with reference to Figure 3, the significant speedup in transitioning from single-buffer processing to circular double-buffer processing comes from using circular double buffers to parallelize the read and write latency for filling and emptying the buffers with the processing latency for generating ML estimates. When the combined read and write latency is shorter than the processing latency (as is the general case), no further benefit is achieved by transitioning to 3, 4, ..., N circular buffers after the latency has been masked by changing from a single buffer to a circular double buffer in the processing loop. Additional buffers may result in further reductions in processing latency if the combined read and write latency is longer than the processing latency, but in practice, the combined read and write latency is usually shorter than the processing latency.

[0115] -Several selected advantages- In one embodiment, the systems and methods described herein present a cyclic double-buffer configuration for multivariate anomaly detection on a real-time stream of values. In one embodiment, the cyclic double-buffer configuration hides read / write delays and achieves greater throughput and shorter end-to-end computation delays than those achievable with a single-buffer configuration.

[0116] In one embodiment, the systems and methods described herein enable the transformation of batch-by-batch multivariate ML anomaly detection to a real-time streaming architecture.

[0117] In one example, a precise framework for separating and analyzing the component contributions to end-to-end delay time in both single-circular buffer ML processing and circular double-buffer ML processing is presented herein.

[0118] -Cloud or Enterprise Implementation- In one embodiment, the System (such as the Streaming ML Estimation System 100) is a computing / data processing system that includes a collection of computing applications or distributed computing applications for access and use by other client computing devices communicating with the System over a network. In one embodiment, the Streaming ML Estimation System 100 is a component of a time-series data service configured to collect, supply, and perform operations on time-series data. The applications and computing system may operate using or be configured to be implemented using a cloud-based network computing system, an Infrastructure-As-A-Service (IAAS), Platform-As-A-Service (PAAS), or Software-As-A-Service (SAAS) architecture, or other types of networked computing solutions. In one embodiment, the System provides at least one or more of the functions disclosed herein, and a graphical user interface for accessing and operating those functions. In one embodiment, the Streaming ML Estimation System 100 is a centralized server-side application that provides at least the functions disclosed herein and is accessed by many users using computing devices / terminals that communicate with the computers of the Streaming ML Estimation System 100 (functioning as one or more servers) over a computer network. In one embodiment, the Streaming ML Estimation System 100 may be implemented by a server or other computing device consisting of hardware and software to implement the functions and features described herein.

[0119] In one embodiment, the components of the streaming ML estimation system 100 may be implemented as a set of one or more software modules executed by one or more computing devices, which are specifically configured for such execution. In one embodiment, the components of the streaming ML estimation system 100 are implemented on one or more hardware computing devices or hosts interconnected by a data network. For example, the components of the streaming ML estimation system 100 may be executed by one or more computing hardware forms such as a central processing unit (CPU), or by network-connected computing devices of general-purpose form, dense input / output (I / O) form, graphics processing unit (GPU) form, and high-performance computing (HPC) form.

[0120] In one embodiment, components of the streaming ML estimation system 100 communicate with each other by electronic messages or signals. These electronic messages or signals may be configured as calls to functions or procedures that access the features or data of the components, such as application programming interface (API) calls. In one embodiment, these electronic messages or signals are transmitted between hosts in a format compatible with the transmission control protocol / internet protocol (TCP / IP) or other computer networking protocols. Components of the streaming ML estimation system 100 may (i) generate or create electronic messages or signals to issue commands or requests to other components, (ii) transmit such messages or signals to other components of the streaming ML estimation system 100, (iii) parse the content of received electronic messages or signals to identify commands or requests that the component can implement, and (iv) automatically implement or execute such commands or requests in response to identifying them. Electronic messages or signals may include queries against a database. Queries may be created and executed in a query language compatible with the database and may be executed in a runtime environment compatible with the query language.

[0121] In one embodiment, a remote computing system may access information or applications provided by the streaming ML estimation system 100, for example, through a web interface server. In one embodiment, the remote computing system may send requests to the streaming ML estimation system 100 and receive responses from it. In one example, access to information or applications may be enabled through the use of a web browser on a personal computer or mobile device. In one example, communication exchanged with the streaming ML estimation system 100 may take the form of, for example, remote REST (Representational State Transfer) requests using JSON (JavaScript Object Notation) as the data exchange format, or simple object access protocol (SOAP) requests to and from an XML server. REST or SOAP requests may include API calls to components of the streaming ML estimation system 100.

[0122] -Software Module Embodiment- Generally, software instructions are designed to be executed by one or more appropriately programmed processors that access memory. These software instructions may include, for example, computer executable code and source code that can be compiled into computer executable code. These software instructions may also include instructions written in interpreted programming languages ​​such as scripting languages.

[0123] In complex systems, such instructions may be placed in program modules, each of which performs a specific task, process, function, or operation. The entire set of modules may be controlled or coordinated in their operation by an operating system (OS) or other form of organizational platform.

[0124] In one embodiment, one or more of the components described herein are configured as modules stored in a non-temporary computer-readable medium. The module consists of stored software instructions that, when executed by a processor accessing at least memory or storage, cause a computing device to perform the corresponding functions described herein.

[0125] -Computing device embodiment- Figure 6 illustrates an exemplary computing system 600, which includes an exemplary computing device configured and / or programmed as a special-purpose computing device, with one or more of the exemplary systems and methods described herein, and / or equivalents. The exemplary computing device may be a computer 605, which includes at least one hardware processor 610, memory 615, and input / output ports 620, operably connected by a bus 625. In one example, the computer 605 may include a streaming ML estimation logic 630, configured to facilitate the generation of a stream of ML estimates in real time from a stream of observations using a circular double buffer, similar to the logic, systems, and methods shown and described with reference to Figures 1, 2, 3, 4, and 5.

[0126] In different embodiments, logic 630 may be implemented in hardware, a non-transient computer-readable medium 637 on which instructions are stored, firmware, and / or a combination thereof. While logic 630 is exemplified as a hardware component mounted on bus 625, it should be understood that in other embodiments, logic 630 may be implemented in processor 610, stored in memory 615, or stored in disk 635.

[0127] In one embodiment, the logic 630 or computer is a means (e.g., a structure such as hardware, a non-temporary computer-readable medium, or firmware) for performing the described action. In some embodiments, the computing device may be a server operating within a cloud computing system, a server configured within a Software as a Service (SaaS) architecture, a smartphone, a laptop, a tablet computing device, etc.

[0128] This means may be implemented as an ASIC programmed to facilitate the generation of a stream of ML estimates in real time from a stream of observations, for example, using a circular double buffer. This means may also be implemented as a stored computer executable instruction that is temporarily stored in memory 615 and then presented to computer 605 as data 640 executed by processor 610.

[0129] The logic 630 may also provide means (e.g., hardware, a non-temporary computer-readable medium for storing executable instructions, firmware) for generating a stream of ML estimates in real time from a stream of observations using a circular double buffer.

[0130] In general, an exemplary configuration of computer 605 is described, where the processor 610 may be a wide variety of processors, including dual microprocessors and other multiprocessor architectures. Memory 615 may include volatile memory and / or non-volatile memory. Non-volatile memory may include, for example, ROM, PROM, etc. Volatile memory may include, for example, RAM, SRAM, DRAM, etc.

[0131] The storage disk 635 may be operably connected to the computer 605, for example, via an input / output (I / O) interface (e.g., a card, device) 645 and an input / output port 620, controlled by at least an input / output (I / O) controller 647. The disk 635 may be, for example, a magnetic disk drive, a solid-state disk drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, a memory stick, etc. Furthermore, the disk 635 may be a CD-ROM drive, a CD-R drive, a CD-RW drive, a DVD-ROM, etc. The memory 615 may store, for example, a process 650 and / or data 640. The disk 635 and / or the memory 615 may store an operating system that controls the computer 605 and allocates its resources.

[0132] Computer 605 interacts with, controls, and / or is controlled by, input / output (I / O) devices via input / output (I / O) controller 647, I / O interface 645, and input / output port 620. Input / output devices may include, for example, one or more displays 670, printers 672 (such as inkjet, laser, or 3D printers), audio output devices 674 (such as speakers or headphones), character input devices 680 (such as keyboards), cursor control devices 682 for pointing and selection input (such as mice, trackballs, touchscreens, joysticks, pointing sticks, electronic styluses, or electronic pen tablets), audio input devices 684 (such as microphones or external audio players), video input devices 686 (such as video and still cameras, or external video players), image scanners 688, video cards (not shown), disks 635, network devices 655, and the like. Input / output port 620 may include, for example, serial ports, parallel ports, and USB ports.

[0133] Computer 605 may operate within a network environment and, therefore, may be connected to network device 655 via I / O interface 645 and / or I / O port 620. Through network device 655, computer 605 may interact with network 655. Through the network, computer 605 may be logically connected to remote computer 665. The networks that computer 605 may interact with include, but are not limited to, LANs, WANs, and other networks.

[0134] In one embodiment, a computer may be connected to a sensor 690 via an I / O port 620 or a network 660 to receive information about the physical state of a machine, device, system, or facility being monitored (collectively, the "Asset"). In one embodiment, the sensor 690 is configured to monitor physical phenomena occurring in or around the Asset. The Asset typically includes any type of machine or facility having components that perform measurable activities. In one embodiment, the sensor 690 may be operably connected to or mounted on or otherwise configured to detect and monitor physical phenomena occurring in or around the Asset. The sensor 690 may be a networked sensor for monitoring any type of physical phenomenon. The network connection between the sensor 690 and the network 660 may be wired or wireless.

[0135] In one embodiment, computer 605 is configured with logic, such as software modules, to collect readings from sensor 690 and store them as observations in a time-series data structure, such as a time-series database. In one embodiment, computer 605 passively receives sensor telemetry readings that are actively transmitted by sensor 690. For example, sensor telemetry readings may be transmitted from sensor 690 to computer 605 in a real-time stream. In one embodiment, the time-series database is stored in a buffer, as described above. In one embodiment, computer 605 polls sensor 690 to retrieve sensor telemetry readings. In one embodiment, computer 605 receives one or more databases of previously collected observations from sensor 690, for example, from storage 635 or from a remote computer 665.

[0136] -Definitions and Other Embodiments- The acts or functions described or claimed herein are not performed by human thought. Any interpretation that any act or function can be performed by human thought is inconsistent with and contrary to this disclosure.

[0137] In another embodiment, the methods described and / or their equivalents may be implemented by computer executable instructions. Thus, in one embodiment, a non-temporary computer-readable / storage medium consists of computer executable instructions containing algorithms / executable applications that, when executed by a machine, cause the machine (and / or associated components) to perform the methods. Exemplary machines include, but are not limited to, processors, computers, servers operating within cloud computing systems, servers configured within a Software as a Service (SaaS) architecture, and smartphones. In one embodiment, a computing device is implemented by one or more executable algorithms configured to perform any of the disclosed methods.

[0138] In one or more embodiments, the disclosed methods or equivalents thereof are implemented by either computer hardware configured to carry out the methods, or by computer instructions embodied in a module stored on a non-temporary computer-readable medium, the instructions being configured as executable algorithms that, when executed by at least the processor of a computing device, carry out the methods.

[0139] For the sake of simplicity of explanation, the methods illustrated in the diagrams are shown and explained as a series of blocks of the algorithm, but it should be understood that the methods are not limited by the order of the blocks. Some blocks may occur in a different order than those shown and explained, and / or simultaneously with other blocks. Furthermore, fewer than all of the illustrated blocks may be used to implement the illustrated methods. Blocks may be combined or divided into multiple actions / components. Furthermore, additional and / or alternative methods may use additional actions not illustrated in the blocks.

[0140] The following includes definitions of selected terms used herein. The definitions include various examples and / or forms of constituents that fall within the scope of the terms and may be used for implementation. Examples are not intended to be limiting. Both singular and plural forms of terms may be within the scope of the definitions.

[0141] References to "one embodiment," "one example," or "one example" indicate that the embodiment or example described in this way may include certain features, structures, characteristics, properties, elements, or limitations, and that not all embodiments or examples necessarily include those specific features, structures, characteristics, properties, elements, or limitations. Furthermore, repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may.

[0142] As used herein, “data structure” is an organization of data within a computing system stored in memory, a storage device, or other computerized system. A data structure may be any one of the following, for example, a data field, a data file, a data array, a data record, a database, a data table, a graph, a tree, a linked list, etc. A data structure may be formed from and include many other data structures (for example, a database includes many data records). Other examples of data structures are similarly possible according to other embodiments.

[0143] When used herein, “computer-readable medium” or “computer storage medium” refers to a non-temporary medium that stores instructions and / or data configured to perform one or more of the disclosed functions when executed. In some embodiments, data may function as instructions. Computer-readable medium can take the form of, but are not limited to, non-volatile and volatile media. Non-volatile media may include, for example, optical disks and magnetic disks. Volatile media may include, for example, semiconductor memory and dynamic memory. Common forms of computer-readable medium may include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, other magnetic media, application-specific integrated circuits (ASICs), programmable logic devices, compact disks (CDs), other optical media, random access memory (RAM), read-only memory (ROM), memory chips or cards, memory sticks, solid-state storage devices (SSDs), flash drives, and other media on which a computer, processor, or other electronic device can function together. Each type of medium may include stored instructions for an algorithm, which, when selected for implementation in one embodiment, is configured to perform one or more of the disclosed and / or claimed functions.

[0144] Where used herein, “logic” refers to components implemented using computer or electrical hardware, a non-transient medium containing instructions for an executable application or program module, and / or a combination thereof, for the purpose of performing any of the functions or actions disclosed herein, and / or for the purpose of performing functions or actions from another logic, method, and / or system as disclosed herein. Equivalent logic may include firmware, an algorithm-programmed microprocessor, a separate logic (e.g., an ASIC), at least one circuit, an analog circuit, a digital circuit, a programmed logic device, a memory device containing instructions for an algorithm, any of which may be configured to perform one or more of the disclosed functions. In one embodiment, logic may include one or more gates, a combination of gates, or other circuit components configured to perform one or more of the disclosed functions. Where multiple logics are described, it may be possible to combine multiple logics into one logic. Similarly, where a single logic is described, it may be possible to distribute a single logic among multiple logics. In one embodiment, one or more of these logics are corresponding structures associated with performing the disclosed and / or claimed functions. The choice of which type of logic to implement can be based on the desired system conditions or specifications. For example, if faster speed is a consideration, hardware may be chosen to implement the function. If lower cost is a consideration, stored instructions / executable applications may be chosen to implement the function.

[0145] An "operable connection," or a connection to which an entity is "operably connected," is one to which signals, physical communications, and / or logical communications can be transmitted and / or received. An operable connection may include physical interfaces, electrical interfaces, and / or data interfaces. An operable connection may include different combinations of interfaces and / or connections sufficient to enable operable control. For example, two entities may be operable connected to communicate signals with each other directly or through one or more intermediate entities (e.g., processors, operating systems, logical, non-transient computer-readable media). Logical and / or physical communication channels may be used to create an operable connection.

[0146] As used herein, “User” includes, but is not limited to, one or more persons, a computer, or other device, or a combination thereof.

[0147] While the disclosed embodiments are illustrated and described in considerable detail, they are not intended to limit or restrict the scope of the appended claims in any way. Naturally, for the purpose of illustrating various aspects of the subject matter, it is impossible to describe all conceivable combinations of components or methods. Therefore, this disclosure is not limited to the specific details or exemplary examples shown and described. Accordingly, this disclosure is intended to encompass variations, modifications, and alterations that fall within the scope of the appended claims.

[0148] To the extent that the terms “including” or “including” are used in a detailed description or claim, they are intended to be comprehensive in a manner similar to how the term “equipped with” is interpreted when used as a transitional term in a claim.

[0149] To the extent that the term "or" is used in the detailed description or claims (e.g., A or B), it is intended to mean "A or B or both." When the applicants intend to indicate "A or B only, but not both," the phrase "A or B only, but not both" is used. Therefore, the use of "or" in this specification is inclusive and not exclusive.

Claims

1. A computer method for generating a stream of estimated values ​​in real time from a stream of observed values, wherein the method is: Receiving observed values ​​from the aforementioned stream of observed values, This includes reading the observed values ​​into a cyclic buffer in real time, wherein the cyclic buffer includes a first buffer and a second buffer configured together in a cyclic configuration. The method further includes a machine learning model generating an estimated value of the expected observed value from the observed value in the cyclic buffer, wherein the generation of the estimated value alternately repeats the process of generating the estimated value from the observed value in the first buffer in parallel while the second buffer is being read, and generating the estimated value from the observed value in the second buffer in parallel while the first buffer is being read. The method described above is a method performed by a computer, further comprising writing the estimated values ​​to a stream of estimated values ​​in real time when the estimated values ​​are generated.

2. A computer method according to claim 1, further comprising generating a warning that an anomaly has been detected in response that one or more residuals between one of the observed values ​​and one of the estimated values ​​of the one observed value satisfy a threshold.

3. The process involves sequentially reading the observed values, generating the estimated values ​​from the observed values, and measuring the end-to-end delay time for writing the estimated values ​​to the stream of estimated values. The end-to-end delay time is compared with a threshold value for the maximum allowable delay time, A method performed by a computer according to claim 1, further comprising switching to reading the observed value into the circular buffer in response to satisfying the threshold for the maximum allowable delay time.

4. The aforementioned method, Further including instantiating a primary compute instance, wherein the primary compute instance is configured to generate the estimates by the machine learning model, The method further includes instantiating an auxiliary computation instance, the auxiliary computation instance being configured to read the observed values ​​into the circular buffer and write the estimated values ​​to the stream of estimated values. The method according to claim 1, further comprising locating the circular buffer in a memory location accessible from both the primary computing instance and the auxiliary computing instance.

5. The observed value corresponding to the estimated value is written to the stream of the estimated value together with the estimated value, The method performed by a computer according to claim 2, further comprising writing the residual corresponding to the estimated value together with the estimated value to the stream of the estimated value.

6. The observed values ​​are a vector containing the observed values ​​for each signal within the set of signals. The aforementioned estimate is a vector containing the estimate for each signal within the set of signals, The aforementioned machine learning model is a multivariate machine learning model. The method performed by a computer according to claim 1, further comprising generating an estimate for one of the signals in the set of signals based on observed values ​​of the signals in the set other than one of the signals.

7. The computer method according to claim 1, further comprising training the machine learning model to generate an estimate of the expected observation based on training observations representing normal operation.

8. Determining the buffer length, The method performed by a computer according to claim 1, further comprising configuring the first buffer and the second buffer to receive a number of observations equal to the buffer length before they become full.

9. The method executed by a computer according to claim 1, wherein the machine learning model is a multivariate state estimation technology model.

10. A program for causing one or more computers to execute the method according to any one of Claims 1 to 9.

11. A computing system comprising one or more computers, wherein the computing system is configured by computer-executable instructions to perform operations including the operations described in any one of claims 1 to 9.

12. On the computer, Receiving observations from a stream of observations, The system reads the observed values ​​into a cyclic buffer in real time, and the cyclic buffer includes a first buffer and a second buffer configured together in a cyclic configuration. The machine learning model is further used to generate an estimated value of the expected observed value from the observed value in the cyclic buffer, and the generation of the estimated value is performed by repeatedly and alternately generating the estimated value from the observed value in the first buffer in parallel while the second buffer is being read, and generating the estimated value from the observed value in the second buffer in parallel while the first buffer is being read. A program that further causes the estimated values ​​to be written to a stream of estimated values ​​in real time as the estimated values ​​are generated.

13. To the aforementioned computer, The process involves sequentially reading the observed values, generating the estimated values ​​from the observed values, and measuring the end-to-end delay time for writing the estimated values ​​to the stream of estimated values. The end-to-end delay time is compared with a threshold value for the maximum allowable delay time, The program according to claim 12, further comprising switching to reading the observed value into the circular buffer in response to the threshold being met.

14. At least one processor connected to at least one memory, A computing system comprising a non-temporary computer-readable medium containing stored instructions for generating a stream of estimates from a stream of observations in real time, wherein, when the instructions are executed by the at least one processor, the computing system Receiving observed values ​​from the aforementioned stream of observed values, The system reads the observed values ​​into a cyclic buffer in real time, and the cyclic buffer includes a first buffer and a second buffer configured together in a cyclic configuration. The machine learning model is further used to generate an estimated value of the expected observed value from the observed value in the cyclic buffer, and the generation of the estimated value is performed by repeatedly and alternately generating the estimated value from the observed value in the first buffer in parallel while the second buffer is being read, and generating the estimated value from the observed value in the second buffer in parallel while the first buffer is being read. A computing system that further causes the estimated values ​​to be written to a stream of estimated values ​​in real time as the estimated values ​​are generated.

15. The instruction is given to the computing system: Further instantiation of a primary compute instance is performed, and the primary compute instance is configured to generate the estimates by the machine learning model. Further instantiation of an auxiliary computation instance is performed, and the auxiliary computation instance is configured to read the observed values ​​into the circular buffer and write the estimated values ​​to the stream of estimated values, The computing system according to claim 14, further comprising the action of locating the circular buffer in a memory location accessible from both the primary computing instance and the auxiliary computing instance.

Images