Creating Hybrid Sensors to Complement Intrusive Sampling

Abstract

A hybrid sensor is created by training a machine learning model, such as a neural network, based on a training dataset. The training dataset includes a first time series of upstream data having a forward dependency on a target variable, a second time series of downstream data having a backward dependency on the target variable, and a time series of measured target variable data related to the target variable. The target variable has a measurement frequency that is lower than the measurement frequencies associated with the upstream sensor data and the downstream sensor data. The hybrid sensor can predict the value of the target variable at a given time, for example, while no actual measured target variable values ​​are available.

Description

[Background technology]

[0001] The present invention relates generally to computers and computer applications, and more specifically to creating hybrid sensors in process industries, such as engineering, manufacturing, or other industrial or industrial processes, or combinations thereof.

[0002] In petroleum, chemical, and food process engineering, for example, operational health depends on full continuous monitoring of physical values ​​such as pressure, temperature, flow rate, or chemical composition or concentration, or a combination of these, during the engineering process. The placement of sensors in the engineering process allows systems implementing such process engineering to monitor such physical variables. However, such monitoring still relies on intrusive sampling and may include time delays before the results of the sensor data are known. For example, monitoring chemical analysis in an industrial process fluid flow relies on intrusive manual sampling and completion offline in a laboratory. Such intrusive manual sampling and lab analysis can create problems, leading to a low number of samples actually sampled and a delay in knowing the current chemical composition of the flow. Both problems make it more likely that opportunities to tune the engineering process for high productivity will be missed. Summary of the Invention

[0003] For example, a system and method for generating a hybrid sensor in industrial processing is provided. The system may include a hardware processor in one aspect. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than the measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and time series of measured target variable data to predict a value of the target variable at a given time.

[0004] In another aspect, the system may include a hardware processor. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than the measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and measured target variable data of a time series to predict a value of the target variable at a given time. The hardware processor may be further configured to generate a causal relationship data structure forward in time linking the upstream sensor data of the first temporal series to the measured target variable data of the time series and backward in time linking the downstream sensor data of the second temporal series to the measured target variable data of the time series.

[0005] In another aspect, the system may include a hardware processor. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and measured target variable data of a time series to predict a value of the target variable at a given time. The hardware processor may further be configured to generate a causal relationship data structure that links the upstream sensor data of the first temporal series forward in time to the measured target variable data of the time series and links the downstream sensor data of the second temporal series backward in time to the measured target variable data of the time series. The upstream sensor data and downstream sensor data may be selected based on the causal relationship data structure.

[0006] In another aspect, the system may include a hardware processor. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The trained machine learning model may predict the value of the target variable at the given time. Based on the predicted value of the target variable for the given time, the hardware processor may further control a set point in an industrial process.

[0007] In another aspect, the system may include a hardware processor. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least the first time series of upstream sensor data within the first time window, the second time series of downstream sensor data within the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The trained machine learning model, such as a neural network, may predict the value of the target variable at the given time. Based on the predicted value of the target variable for the given time, the hardware processor may further control a set point in an industrial process. The industrial process may include oil sands processing, and the target variable may include a characteristic associated with oil sands processing tailings.

[0008] In another aspect, the system may include a hardware processor. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and measured target variable data of a time series to predict a value of the target variable at a given time. The machine learning model, such as a neural network, may include an aggregation of a first neural network trained to predict the value of the target variable based on a forward prediction of the upstream sensor data of the first time series, a second neural network trained to predict the value of the target variable based on a backward prediction of the downstream sensor data of the second time series, and a third neural network trained to predict the value of the target variable based on the measured target variable data of the time series.

[0009] In another aspect, the system may include a hardware processor. A storage device may be coupled to the hardware processor. The hardware processor may be configured to receive a first time series of upstream sensor data having a forward dependency on a target variable. The hardware processor may also be configured to receive a second time series of downstream sensor data having a backward dependency on the target variable. The hardware processor may also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The hardware processor may also be configured to determine a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The hardware processor may also be configured to determine a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The hardware processor may also be configured to train a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and measured target variable data of a time series to predict a value of the target variable at a given time. The machine learning model, such as a neural network, may include aggregating a first neural network trained to predict the value of the target variable based on a forward prediction of the upstream sensor data of the first time series, a second neural network trained to predict the value of the target variable based on a backward prediction of the downstream sensor data of the second time series, and a third neural network trained to predict the value of the target variable based on the measured target variable data of the time series.A report time of a measured target variable has a delay from a collection time of the target variable, and the training data set including the time series of measured target variables can include data for the measured target variable determined at a report time and shifted to a corresponding collection time of the measured target variable, and the machine learning model, such as a neural network, can predict the value of the target variable at the given time without the delay at run time.

[0010] A method for generating a hybrid sensor in industrial processing, in one aspect, can include receiving a first time series of upstream sensor data having a forward dependency on a target variable. The method can also include receiving a second time series of downstream sensor data having a backward dependency on the target variable. The method can also be configured to receive a time series of measured target variable data associated with the target variable, the target variable having a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The method can also include determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The method can also include determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The method can also include training a machine learning model based on a training dataset including at least the upstream sensor data of the first time series in the first time window, the downstream sensor data of the second time series in the second time window, and the measured target variable data of the time series to predict a value of the target variable at a given time.

[0011] In another aspect, a method of generating a hybrid sensor in industrial processing can include receiving upstream sensor data of a first time series having a forward dependency on a target variable. The method can also include receiving downstream sensor data of a second time series having a backward dependency on the target variable. The method can also include receiving a time series of measured target variable data associated with the target variable, the target variable can have a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The method can also include determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The method can also include determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The method can also include training a machine learning model based on a training dataset including at least upstream sensor data of the first time series in the first time window, downstream sensor data of the second time series in the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The method may further include generating a causal data structure forward in time linking the upstream sensor data of the first temporal series to the time series of measured target variable data and backward in time linking the downstream sensor data of the second temporal series to the time series of measured target variable data.

[0012] In yet another aspect, a method for generating a hybrid sensor in industrial processing can include receiving upstream sensor data of a first time series having a forward dependency on a target variable. The method can also include receiving downstream sensor data of a second time series having a backward dependency on the target variable. The method can also include receiving a time series of measured target variable data associated with the target variable, the target variable can have a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The method can also include determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The method can also include determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The method can also include training a machine learning model based on a training dataset including at least upstream sensor data of the first time series in the first time window, downstream sensor data of the second time series in the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The method may further include generating a causal relationship data structure forward in time linking the upstream sensor data of the first temporal series to the time series of measured target variable data and backward in time linking the downstream sensor data of the second temporal series to the time series of measured target variable data, wherein the upstream sensor data and the downstream sensor data may be selected based on the causal relationship data structure.

[0013] In yet another aspect, a method for generating a hybrid sensor in industrial processing can include receiving upstream sensor data of a first time series having a forward dependency on a target variable. The method can also include receiving downstream sensor data of a second time series having a backward dependency on the target variable. The method can also include receiving a time series of measured target variable data associated with the target variable, the target variable can have a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The method can also include determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The method can also include determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The method can also include training a machine learning model based on a training dataset including at least upstream sensor data of the first time series in the first time window, downstream sensor data of the second time series in the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The trained machine learning model, such as a neural network, can predict the value of the target variable at the given time, and the hardware processor can further control a set point in an industrial process based on the predicted value of the target variable for the given time.

[0014] In another aspect, a method for generating a hybrid sensor in industrial processing can include receiving upstream sensor data of a first time series having a forward dependency on a target variable. The method can also include receiving downstream sensor data of a second time series having a backward dependency on the target variable. The method can also be configured to receive a time series of measured target variable data associated with the target variable, the target variable can have a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The method can also include determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The method can also include determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The method can also include training a machine learning model based on a training dataset including at least upstream sensor data of the first time series in the first time window, downstream sensor data of the second time series in the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The trained machine learning model, such as a neural network, can predict the value of the target variable at the given time. Based on the predicted value of the target variable for the given time, the hardware processor can further control a set point in an industrial process. The industrial process can include oil sands processing and the target variable can include a characteristic associated with tailings.

[0015] In another aspect, a method of generating a hybrid sensor in industrial processing can include receiving upstream sensor data of a first time series having a forward dependency on a target variable. The method can also include receiving downstream sensor data of a second time series having a backward dependency on the target variable. The method can also include receiving a time series of measured target variable data associated with the target variable, the target variable can have a lower measurement frequency than a measurement frequency associated with the upstream sensor data and the downstream sensor data. The method can also include determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. The method can also include determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. The method can also include training a machine learning model based on a training dataset including at least upstream sensor data of the first time series in the first time window, downstream sensor data of the second time series in the second time window, and time series of measured target variable data to predict a value of the target variable at a given time. The machine learning model, such as a neural network, can include an aggregation of a first neural network trained to predict the value of the target variable based on a forward prediction of the upstream sensor data of the first time series, a second neural network trained to predict the value of the target variable based on a backward prediction of the downstream sensor data of the second time series, and a third neural network trained to predict the value of the target variable based on measured target variable data of the time series.

[0016] In one or more aspects of the systems and / or methods disclosed herein, the first time series of upstream sensor data can be one-dimensional or multi-dimensional time series data. In one or more aspects of the systems and / or methods disclosed herein, the second time series of downstream sensor data can be one-dimensional or multi-dimensional time series data. In one or more aspects of the systems and / or methods disclosed herein, the machine learning model can be a neural network or a neural network model.

[0017] A computer-readable storage medium storing program instructions executable by a machine for performing one or more of the methods described herein may also be provided.

[0018] Further features, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings, in which like reference numbers indicate identical or functionally similar elements. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram of different sets of timing diagrams and measurements operable in implementing a hybrid sensor in accordance with an embodiment of the system. [Diagram 2] FIG. 2 illustrates tailings sampling for the oil sands industry as an exemplary industrial process in one embodiment. [Diagram 3] FIG. 3 illustrates correlated metrics for a sands oil tailings example according to an embodiment. [Figure 4] FIG. 4 shows another diagram illustrating the different frequencies at which different sets of sensor data are acquired in an embodiment. [Diagram 5] FIG. 5 illustrates system components in an embodiment. [Figure 6]FIG. 6 shows a causal processing graph in an embodiment. [Figure 7] FIG. 7 is another diagram illustrating the temporal plane relative to the sources of data used in predicting target sensor values ​​in an embodiment. [Figure 8] FIG. 8 illustrates a dynamic graph for forward and backward estimation in an embodiment. [Figure 9] FIG. 9 illustrates a process flow of upstream, target, and downstream variables in determining one or more hidden sensor values ​​in an embodiment. [Figure 10] FIG. 10 illustrates an example neural network architecture that can be trained to predict a target variable at a given point in time. [Figure 11] FIG. 11 is a flow diagram illustrating a method in an embodiment. [Figure 12] FIG. 12 is another flow diagram illustrating a method in accordance with an embodiment. [Figure 13] FIG. 13 is a diagram illustrating components of a system that can produce a hybrid sensor in one embodiment, by way of example. [Figure 14] FIG. 14 illustrates a schematic of an example computer or processing system that may be used to implement the hybrid sensor system in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] In aspects, systems, devices, and methods are disclosed that can create hybrid sensors. For example, the systems, devices, and methods can create hybrid sensors to supplement expensive intrusive sampling with correlated high frequency sampling from non-intrusive sensors in industrial processes, such as manufacturing, engineering, chemical, and other industrial processes. In aspects, the hybrid sensors in embodiments can provide a technical improvement over one or more existing sensor devices or manual sampling that cannot be used to measure data at a relatively high frequency at a desired rate for acquiring sensor data, for example.

[0021] A system capable of generating a hybrid sensor may include, for example, computer-implemented components implemented or operated on, or coupled to, one or more processors or hardware processors or both, as examples. A method for generating a hybrid sensor may be executed or operated on such one or more hardware processors. The one or more hardware processors may include components such as, for example, programmable logic devices, microcontrollers, memory devices, or other hardware components, or combinations thereof, that may be configured to perform the respective tasks described in this disclosure. The associated memory device may be configured to selectively store instructions executable by the one or more hardware processors.

[0022] The processor may be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), other suitable processing components or devices, or a combination of one or more thereof. The processor may be coupled to a memory device. The memory device may include a random access memory (RAM), a read only memory (ROM), or another memory device, and may store data and / or processor instructions for implementing various functions associated with the methods and / or systems described herein. The processor may execute computer instructions stored in the memory or received from another computer device or medium.

[0023] By placing sensors in an engineering or other industrial process, the system in an embodiment can use the sensor device to continuously monitor a physical variable, for example, in a high frequency, non-intrusive, in-line sampling at fixed intervals. In another aspect, the system can receive or acquire the continuously monitored physical variable.

[0024] In an embodiment, the system can generate a hybrid sensor, which allows for predicting a target quantity in unmeasured space between any two consecutive target variable measurements that are considered to be separated by a relatively long interval, for example. In an embodiment, the system can generate a causal diagram or data structure for deriving a hybrid sensor that fuses signals from multiple sources in a manner that utilizes a causal diagram or data structure. Hybrid sensors can be implemented for various processing and manufacturing industries, for example, where composition, purity, or any quality related quantity or variable is the target variable in the query. The system can derive, for example, timely and accurate predictions of quantities that were not previously available as frequently as desired. Hybrid sensors can provide finer resolution and earlier availability of target quantities, allowing for better proactive action. For example, based on the predicted unobserved target quantities, the system can automatically or autonomously control one or more set points, for example, automatically or autonomously change one or more set points in an industrial process.

[0025] FIG. 1 is an illustration of a timing diagram and different sets of measurements that an embodiment system can operate in implementing a hybrid sensor. The system can operate in embodiments using different types or sets of measurements. The measurements provide variables or quantities, such as, for example, temperature, pressure, constituent amounts, or other, or combinations thereof. The set of upstream measurements can be measured by one or more sensors at a relatively high frequency, for example, in-line sensing at another time interval, which can be every 5 minutes or a fixed interval. For example, one or more sensors measure the upstream data in real time or near real time. The time series of the upstream measurements is shown as a series of ellipses in FIG. 1. There can be many different sets of downstream measurements. The set of downstream measurements can be measured by one or more sensors at a relatively high frequency, for example, in-line sensing at another time interval, which can be every 5 minutes or a fixed interval. For example, one or more sensors measure the downstream data in real time or near real time. The time series of the downstream measurements is shown as a series of 'x's in FIG. 1. A target measurement for a corresponding target variable or quantity, which may be measured by one or more sensors or determined analytically, is available only slower or less frequently (e.g., every 12 hours) than the set of upstream measurements and the set of downstream measurements. A time series of target measurements taken over time is shown as a series of diamonds in FIG. 1.

[0026] In an embodiment, a model-based hybrid sensor driven by a causal diagram can estimate an unobserved target variable 110 at any time “t” 102, where the system operates on evidence (measured sensor data) from three sources: upstream finely measured covariates 104, e.g., representing sensor measurements from a time window before time “t” 102 (the time interval immediately before and to the left of time “t” 102), downstream finely measured covariates 106, e.g., representing sensor measurements from a time window after time “t” 102 (the time interval immediately after and to the right of time “t” 102), and the most recently available measurement of the target variable 108. Upstream measurements measure upstream materials or conditions, e.g., raw materials input into an industrial process; downstream measurements measure downstream materials or conditions, e.g., by-products, intermediate products, products, or operating conditions, or a combination thereof, obtained during the process. In an embodiment, a model-based hybrid sensor driven by a causal diagram can be implemented as a neural network architecture, where a neural network is trained to, for example, predict an unobserved target variable 110 between two observed target variables 108, 112.

[0027] In brief, an artificial neural network (ANN) or neural network (NN) is a machine learning model that is trained to predict or classify input data. An artificial neural network may include successive layers of neurons that are interconnected, and the output signals of neurons in one layer are weighted and transmitted to neurons in the next layer. A neuron Ni in a given layer may be connected to one or more neurons Nj in the next layer, and a different weight wij is associated with each neuron-neuron connection Ni-Nj to weight the signal transmitted from Ni to Nj. A neuron Nj generates an output signal based on its integrated input, and the weighted signal may be propagated through successive layers of the network from the input to the output neuron layer. An artificial neural network machine learning model undergoes a training phase during which a set of weights associated with each neuron layer is determined. The network is provided with a set of training data in an iterative training scheme, where the weights are iteratively updated as the network “learns” from the training data. The resulting trained model can be applied to perform a task based on new data with the weights defined via the training operation.

[0028] For example, an industrial process includes one or more sensors that measure one or more corresponding variables or quantities that are set or input. An example of an industrial process can be an oil sands process that includes bitumen extraction from oil sands deposits, which can include bitumen upgrading to give diluted bitumen, synthetic crude oil, and refining of the crude oil to allow for the production of end products such as gasoline, diesel, lubricants, and asphalt. During such industrial processing, measurements or sampling are performed to monitor the quality of the process, intermediates, or end products produced, or both. Obtaining some of these measurements, however, can be intrusive and may not be available as frequently as desired. In another aspect, samples that require lab processing can impose a delay in obtaining measurements, for example, depending on the speed of the lab processing. A hybrid sensor in an embodiment can, for example, measure or predict such unobserved measurements in near real time and allow for correcting problems that may be detected in the process or improving the quality of intermediates or end products, or both, obtained during the process, or a combination thereof.

[0029] FIG. 2 illustrates tailings sampling for the oil sands industry as an example industrial process in one embodiment. Tailings sampling analysis can take up to two days. Tailings are a mixture of water, sand, clay, and the remainder bitumen, and are a by-product of the hydrothermal treatment process 204 used to separate the bitumen from the sand and clay. Tailings are accumulated in large engineered dam and dike systems called tailing ponds 202. Tailings ponds 202 are sediment basins that allow the process water to be separated and continuously recycled. Water is continuously recycled from the tailings ponds back to the extraction process, reducing the use of fresh water from rivers and other sources. Oil sands producers recycle 78-86% of the water used. Tailings can have impacts on the environment and also on the appearance of the refinery. The tailings composition is associated with high sampling frequency (in-line) non-intrusive sensors 206 corresponding to variables such as upstream raw materials, and downstream measurements or sensor values ​​208 corresponding to downstream material resulting from hydrothermal processing (post-primary processing) (e.g., measured every 5 minutes). In an embodiment, a hybrid sensor 210 can measure the tailings during observed sampling (e.g., during sample analysis) and provide a continuous measurement of the tailings.

[0030] FIG. 3 illustrates associated metrics for a sands oil tailings sample in an embodiment. A raw material 302 is treated with hot water in a bitumen primary treatment process 304. Various sensors measure upstream sensor data such as raw material content measurements, inflows, and process variables 310. The hot water treatment process refines tailings (by-products) 306. The remainder of the bitumen mixture (non-tailings) is treated in a bitumen secondary treatment 308, such as centrifugation. In the post-hot water treatment, various sensors also measure downstream sensor data or post-treatment content measurements and outflow variables 312. In this example, the target variable 314 is a measurement associated with the tailings sample, such as the content of the tailings sample produced as a by-product (or intermediate variable). Such measurements are typically taken less frequently than the upstream sensor metrics 310 and downstream sensor metrics 312 and incur expensive intrusive sampling, such as offline lab analysis. Both the upstream metric 310 and the downstream metric 312 are frequently sampled non-intrusive sensors / metrics that are sampled, for example, at a higher frequency than the sampling of the target variable.

[0031] 4 shows another diagram illustrating different frequencies at which different sets of sensor data are acquired in an embodiment. Upstream sensor metrics 402 can be acquired at a fixed time interval, e.g., a first frequency. Downstream sensor metrics 404 can be acquired at another fixed time interval, e.g., a second frequency. A target variable 406 for which it is desired to predict unobserved or unmeasured values ​​can be a variable or quantity that is measured less frequently than the first and second frequencies.

[0032] In an embodiment, the system or method or both combine actual monitored less frequent values ​​with predicted more frequent values ​​to create a hybrid sensor. The hybrid sensor allows for estimating and storing values ​​for variables at a higher rate comparable to big data analysis of intrusive, less frequent measurements; or else more frequent, non-intrusive measurements. Comparable and consistent fine-grained frequency measurements allow for better data-driven decision making and process control for optimal recovery and yield with acceptable quality. In an embodiment, the hybrid sensor can allow for detecting possible environmental damage before a possible catastrophe or disaster, e.g., due to cumulative effects. In another aspect, the hybrid sensor can allow for identifying any large losses of expensive, economical raw materials sooner.

[0033] FIG. 5 illustrates system components in an embodiment. The illustrated components include one or more processors or hardware processors executing computer processes or threads or both, for example, as described above. The system of an embodiment can use backward in time estimation using downstream (e.g., fine) measurements and forward in time estimation using upstream (e.g., fine) measurements and past own (e.g., coarse) measurements in identifying one or more hidden values ​​for a hybrid sensor. By way of example, the storage device 502 can store upstream measurement data (or metrics), such as sensor measurement data, downstream measurement data (or metrics), such as sensor measurement data, and historical data including data of the hybrid sensor. As described above, the upstream measurement data can include data related to inflow materials or conditions or both in a process (e.g., industrial, engineering, chemical process). The downstream measurement data can include data related to effects on the process resulting from processing of inflow materials or conditions or both. The hybrid sensor data can include data related to variables or attributes for which the hybrid sensor predicts unobserved values ​​at points in time, such as between actually measured metrics. The hybrid sensor data can include historically actual measured data of variables or attributes and can also include previously predicted values. The upstream and downstream data can be acquired or collected by sensors performing high frequency (e.g., higher frequency than the hybrid sensor data) sampling of industrial processes, such as real-time physical or chemical processing. The collected sensor results can be stored, for example, instantly, in real-time or near real-time, back to a repository (e.g., storage device 502).

[0034] The computer processor may receive or obtain, for example, upstream data 504, downstream data 508, and hybrid sensor data 506 from a storage device 502 that stores such a historical database. The computer processor may also receive a time point of prediction. In an embodiment, a time window before the time point of prediction and a time window after the time point of prediction are determined. In subsequent processing, upstream data measured within the time window before the time point of prediction and downstream data measured within the time window after the prediction may be used. The time window before the time point of prediction is also referred to as a first time window for simplicity of explanation. The time window after the time point of prediction is also referred to as a second time window for simplicity of explanation. In an embodiment, the received data may be data within these time windows.

[0035] At 510, the computer processor constructs an in-memory data structure, e.g., a causality graph or causality processing graph, which represents the causal relationships or relations between measurements, e.g., upstream and downstream metrics. The causality graph or causality processing graph (also referred to as a qualitative causality graph) may include specific upstream and downstream variable names or identifiers or both: this information may be received as input. In an embodiment, the block indicated at 510 includes a corresponding graph data structure using an embodiment of the method disclosed herein, which also includes a data structure capturing this qualitative information in the form of accompanying sensor measurements with corresponding time series data or respective variables under inquiry, i.e., which are difficult to detect / measure and whose values ​​can be predicted by extrapolation of the qualitative graph, and a computational data structure corresponding to the temporal series of measurements.

[0036] In an embodiment, the computer processor may perform feature selection 512 to identify upstream and downstream sensors that have a strong relevant relationship to the target variable. Other feature selection methods may be used that use statistics to evaluate the relationship between each input variable and the target variable and select the input variables that have the strongest association with the target variable. The selected features may include data that may be determined to be relevant to determining or predicting the hybrid sensor data. Examples of features may include material content and operating conditions such as temperature, pressure, and other operating conditions. For example, using the example shown in Figures 2 and 3, the features may include raw material content measurements, inflow and process variables, post processing content measurements, and outflow variables. Hybrid sensor data may also be selected as features, such as tailings sample data. In an embodiment, feature selection 512 calculates parameters "tau_1" (or referred to as \tau_1), "tau_2" (or referred to as \tau_2), which are lag and lead parameters that correspond to the graph data structure. As an example, these parameters are calculated based on time series of measurement data that are held or stored within a graph data structure.

[0037] In an embodiment, the computer processor constructs and uses in-memory causal processing graph data (e.g., an internal computer data structure for computer processing) and identifies upstream and downstream sensors for selection based on the causal processing graph data structure.

[0038] In an embodiment, the computer processor can predict the hidden value of the hybrid sensor by forward estimation or prediction using upstream data 514, self estimation or prediction using hybrid sensor data 516, and backward estimation or prediction using downstream data 518. The computer processor aggregates the predicted hidden values ​​at 520. In an embodiment, the components shown at 514, 516, 518, and 520 can include neural networks or aggregations of neural networks. The final value can be used for, for example, an application 522. The application 522 can include providing the hybrid sensor, providing an environmental hazard release warning, providing a productivity warning, or another application. In an aspect, the hybrid sensor can be coupled to a physical system and a controller can be activated to change one or more inputs, such as the content or operating conditions or both of a material used in the physical system. For example, a valve or conduit that controls the content amount of a material added to the physical system can be controlled to open or close to control the amount. As another example, an operating condition can be controlled to set a set point, such as temperature, pressure, or another condition or combination thereof. In an embodiment, the hybrid sensor values ​​so determined can automate the control of a physical system in real time or near real time, for example, by a computer processor in direct communication with a controller of the physical system. The determined hybrid sensor values ​​can also be stored in storage device 502, for example, in real time or near real time. The stored hybrid sensor values ​​can be made available to application 522.

[0039] 6 shows a causal processing graph in an embodiment. Upstream process variables (upstream sensors or features) 602 that have an influence on a target variable 604 are linked in a data structure. The target variable 604 is also linked to downstream process variables 606 that are influenced by the target component. In an embodiment, upstream process variables 602 that influence downstream process variables 606 can also be linked.

[0040] In an embodiment, causal relationships are influenced by industrial process sequences, such as sequences of physical and chemical processes. tau_1, tau_2, and tau_3 are values ​​greater than zero and represent time lags for the influences. One or more of these lags can be used as windows in a time series for analysis, such as determining or predicting an unobserved hybrid sensor. For example, the time period from time point t-tau_1 to time point t, where time point t is the time at which an unobserved hybrid sensor is predicted, can be observed during which the upstream process influence on the target variable can be observed due to the relationship between the upstream process variable to the target variable during that time period (also referred to as the first time window described above).

[0041] The downstream process variable 606 is observed during the time period from time point t to time point t+tau_2 (also referred to as the second time window discussed above). In an embodiment, tau_3 is greater than or equal to tau_1, tau_3 is greater than or equal to tau_2, and tau_3 may be equal to tau_1+tau_2.

[0042] FIG. 7 is another diagram illustrating a time horizon associated with sources of data used in predicting target sensor values ​​in an embodiment. The sources of data include upstream data (x_u) 702 acquired or monitored for a time interval from t-tau_1 to t, downstream data (x_d) 704 acquired or monitored for a time interval from t to t+tau_2, and a previously monitored target value (y(t)) 706. A target variable 708, e.g., a hybrid sensor value, is predicted at time t. A system, e.g., a computer processor of the system, generates a near real-time hybrid sensor. In an aspect, the maximum delay of the sensor determined at time point t is tau_2. The value of tau_2 is small, e.g., 2 minutes. In such an embodiment, the time interval between downstream sampling can be smaller than tau_2.

[0043] 8 illustrates another dynamic graph for forward and backward estimation in an embodiment. The time series of upstream data 802 has a forward estimation relationship with the time series of target variable 804 and the time series of downstream data 806, e.g., 808 and 810 from a previous time step to a next time step. In the time series of target variable 804, the target variable has a relationship with the next target variable in the next time step, e.g., as shown at 812. The time series of downstream data 806 has a backward estimation relationship with the time series of target variable, e.g., the downstream data in the next time step has a relationship with the target variable in the previous time step, e.g., as shown at 814.

[0044] FIG. 9 is a diagram illustrating a process flow of upstream, target, and downstream variables in determining one or more hidden sensor values ​​in an embodiment. The process flow uses a dynamic graph in an embodiment. The upstream variables or data or sensor data 902 include one or more upstream variables or features, e.g., U1...Uk, where k refers to a number of different upstream variables. Although not shown, for each different upstream variable, a time series of the upstream variables (e.g., t to t-tau_1) can be used. The target variable 904 in an embodiment is a variable, e.g., y, whose value is predicted at time t, y(t). The downstream variables or data or sensor data 906 can include one or more downstream variables or features, e.g., D1...Dl, where l (the letter el) refers to a number of different downstream variables. Although not shown, a time series of the downstream variables (e.g., t to T+tau_2) can be used for each different downstream variable.

[0045] Figure 10 illustrates an embodiment neural network architecture that can be trained to predict a target variable at a given time point. For example, the trained neural network can function as a hybrid sensor.

[0046] Neural Networks (f u ) 1002 performs upstream prediction. Using the upstream data and the ground truth of the target variables affected by the upstream data, a neural network (f u) is trained to predict the target variable at any given time point given feature values ​​of upstream data (e.g., time series). In an embodiment, neural network 1002 computes intermediate features, i.e., values ​​output by the neural network. The intermediate features (or values ​​generated by the neural network) capture the separable influence on the target due to the lagged values ​​of the upstream variables alone. The input to neural network 1002 is a historical window for each upstream variable (XU1, XU2,...). For each of these upstream variables, e.g., XU_1, the input to neural network 1002 is (t-tau Dmax -tau U1 ), (t-tau Dmax ), i.e., (t-tau Dmax -tau U1 ), (t-tau Dmax ) can be consecutive values ​​over a window ending in

[0047] The input to the neural network is XU1(t-tau Dmax -tau U1 )... XUk(t-tau Dmax -tau Uk ). In an embodiment, the input to the neural network is a historical window of a predetermined length (such as W>=1) for each upstream variable (XU1, XU2,...). For each of these upstream variables, say XU_k, the input can be (t-tau Dmax -tau U1 ), (t-tau Dmax ) the time series data that ends, i.e. (t-tau Dmax -tau U1 ), the values ​​may be consecutive over a window ending at t. In an embodiment, each training point corresponds to a point in time at which the target variable was measured, e.g., if t *If the targets are time points measured in the training data, then (t-tau Dmax =t * ) and the above-mentioned time t * All the different windows of input relative to the training points t can be implied by the above description. In an embodiment, the procedure * Since we choose t, the procedure also has measurements of all downstream variables, which means that * After (tau Dmax +t * After such training, the procedure calculates the time over (t-tau) if the current time is t. Dmax ) to generate a prediction for the target variable. The output of the neural network 1002 is the predicted target variable.

[0048] Here, "t" is used to indicate the current time when the system and / or method in one embodiment described herein is actually used. Relative to real time "t", the system uses time (t-tau Dmax ) relative to such a time point, i.e., time (t-tau Dmax ), the system is guaranteed to have all downstream variables at their respective "lead" times, but this is most limiting if the "lead" time is tau Dmax The system is based on the downstream variables (t - tau Dmax ), the system has access to all measured values ​​of the downstream variables.

[0049] tau Dmaxrefers to the time window of maximum lag between the time point at which the target variable is predicted and tau_2 (the end of the sampling time point for the downstream data).

[0050] Any time, such as "tau" or "lead" relative to \tau, refers to a time to the right (later or in the future) relative to the time \tau.

[0051] A "lag" relative to any time, \tau, refers to a time to the left (earlier or in the past) relative to the time \tau.

[0052] \tau_{U_k}: All of these are “lags” because the influential / causal upstream variables are lagged values ​​from the past. “U” refers to upstream. U1 to Uk refer to k different upstream data, for example, k number of upstream data can be used, each with a time series of values.

[0053] \tau_{D_k}: All of these are “leads” because the influential / causally related downstream variables are late / leading values ​​from relatively future. “D” refers to downstream.

[0054] Here, "t" refers to the present time, e.g., when the system and / or method disclosed herein is actually used. Relative to real time "t", the system is in a state of time (t-tau Dmax ) can generate predictions for the target variable at time (t-tau Dmax ), the system uses lagged values ​​of the upstream variables. For the upstream variables, U_1, this is the predicted time point of the variable U_1, i.e., time (t-tau Dmax ), where the lag is tau_{U_1}: thus, a timestamp is t-tau Dmax - tau U1As mentioned above, for each of these upstream variables, say XU_1, the input is (t-tau Dmax - tau U1 ), (t-tau Dmax A window of data for a time series of continuous measurements ending at (t-tau Dmax ) can be consecutive values ​​over a window ending in

[0055] Neural Networks (f T ) 1004 performs self-prediction. By using the ground truth of the target variable and its influence on the next target variable, the neural network (f T ) is trained to estimate, i.e., predict, the next target variable value for a given target variable value. The input to the neural network 1004 is y(t-tau Dmax - tau_{y}), where "tau_{y}" represents the previous most recent measurement timestamp in the past for the coarsely measured target variable "y". This is also a window of consecutive past measurements of length W (>=1) that ends at timestamp tau_{y}. If W=1, it is the very most recent measurement. If W=2, it is the second-closest from the past, etc. W is a modeling parameter. W can be preselected or preconfigured as a configuration parameter. The output of the neural network 1004 is the predicted target variable.

[0056] Neural Networks (f D ) 1006 performs downstream prediction. Using the ground truth of the downstream data and the target data that influences the downstream data (backward estimation), a neural network (f D ) 1006 can be trained to predict, i.e., estimate, a target variable given future downstream data. The input to the neural network 1006 is Dmax - TauDl ), …, xD1(t-tau Dmax - tau Dl ), where D1···Dl (the letter ell) refers to the number of distinct downstream variables, 1···l (the letter ell).

[0057] Neural network (g) 1008 aggregates neural networks 1002, 1004, and 1006 to output a prediction of an aggregated target variable. For example, neural network (g) may be trained based on the ground truth values ​​of neural networks 1002, 1004, and 1006 to output an aggregated target value. For example, neural network (g) 1008 may output a prediction of y(t-tau Dmax ) can be predicted.

[0058]

number

[0059]

number

[0060] For example, Tau Dmax is the end time point for the window of the time series of downstream data. Neural network (g) 1008, for example, functions as a hybrid sensor.

[0061] Here, "t" refers to the current time "t" since it pertains to a system that is used in real time, e.g., real time. Relative to real time "t", the system uses time (t-tau Dmax ) to generate predictions for the target variable.

[0062] FIG. 11 is a flow diagram illustrating a method of an embodiment. A processor or hardware processor can execute the method of the embodiment. At 1102, the method can include predicting an upstream lag window length (e.g., tau_1) associated with the forward dependency of the target variable measured at the time the target variable was sampled. In an embodiment, the lag and lead can be predicted using known standard time series statistical techniques. As an example, cross-correlation is calculated between the target and each upstream variable for different values ​​of lag (e.g., 1, 2, 3, 4 ...) and the lag corresponding to the highest cross-correlation between the upstream variable and the target is selected as the "lag" parameter. In yet another approach, the system can leave this "lag" parameter as a model hyperparameter over a range of lags, e.g., [1, 2, 3, ..., L], and use a validation set and hyperparameter tuning to select the optimal lag value for each upstream variable. "Lag" relative to any time, e.g., tau, refers to time to the left (forward or past) relative to time tau. \tau\{U_k}: All of these values ​​are "lagged" because the influential / causal upstream variables are lagged values ​​from the past.

[0063] At 1104, the method includes predicting a downstream lag window length (e.g., tau_2) associated with the backward dependency of the target variable measured at the time the target variable was taken. In an embodiment, the lag and lead can be predicted using known standard time series statistical techniques. By way of example, cross-correlation is calculated between the target and each downstream variable for the leads (e.g., 1, 2, 3, 4 ...) and the lead corresponding to the highest cross-correlation between the downstream variable and the target is selected as the "lead" parameter. In yet another approach, the system can leave this "lead" parameter as a model hyperparameter over a range of leads, e.g., [1, 2, 3, ..., L], and use a validation set and hyperparameter tuning to select the optimal lead value for each downstream variable. A "lead" relative to any time, e.g., tau, refers to a time to the right (later or future) relative to time tau. tau_{D_k}: All of these values ​​are “lead” because the influential / causal downstream variables are relatively lag / lead values ​​from the future.

[0064] At 1106, the method includes combining measurements of the upstream variable from a previous time window (from the time the target variable was taken) whose interval (length) from 1102 is tau_1. The upstream variable includes measurements that are finer or more frequent than the measurement frequency of the target variable for which the hybrid sensor is generated and predicted. This time window is also referred to as a first time window. In an embodiment, derivation of various time series features can also be performed over the first time window. Known techniques can be used to derive time series features from a window of time series having one or more successive measurements.

[0065] At 1108, the method includes combining measurements of the downstream variable from an immediately following time window (from the time the target variable was taken) whose interval (length) from 1104 is tau_2. The downstream variable includes measurements that are finer or more frequent than the measurement frequency of the target variable for which the hybrid sensor is generated to predict. This time window is also referred to as a second time window. In an embodiment, derivation of various time series features can also be performed over the second time window. Known techniques can be used to derive time series features from a window of time series having one or more successive measurements.

[0066] At 1110, the method may include learning (training) a mapping from "inputs" spanning combinations of feature measurements 1106-1108 to "outputs" (target variables) that are hybrid sensors for less frequently measured variables (e.g., lab measurement variables that may have delays) using supervision and / or ground truth available in historical data of measurements (e.g., lab measurements). In the case of an embodiment where there is a delay in time from acquisition to obtaining the measurement data, e.g., due to lab processing time, these "output" variables may be aligned in time to the corresponding historical acquisition epochs. In an embodiment, the learning may include training one or more neural networks, e.g., as shown in FIG. 10.

[0067] For historical learning in this embodiment, the processing at 1106 and 1008 respectively considers the upstream and downstream window sizes for all historical acquisition epochs having a corresponding target variable, e.g., sequenced and reported lab data for a variable of interest (e.g., a target variable for which a hybrid sensor is to be generated), resulting from the processing at 1102 and 1104 respectively.

[0068] At 1112, the learned mapping or neural network can be operated at run-time, e.g., at any time point (e.g., “t_any”), by configuring upstream and downstream windows of length tau_1 and tau_2 relative to the time point (t_any − tau_2) to obtain a prediction of the hybrid sensor at the time point (t_any − tau_2). While this example shows a simple case scenario where there is one upstream and one downstream variable, the method can operate with multiple upstream variables or multiple downstream variables or both. In an embodiment, the various “lags” and “leads” can be different for each particular lag (upstream) or lead (downstream) variable. The values ​​used to train the neural network and for estimation by the neural network can be the same values ​​for these “lags” and “leads” corresponding to different corresponding upstream and downstream variables, respectively.

[0069] The method can provide finer-grained frequency of predictions of a slowly measured variable of interest (target variable). For example, at any point in time series, say t_any, the method can predict the target variable for the previous instance in time, i.e. (t-tau_2).

[0070] Referring back to FIG. 4, for example, in an example scenario where a target variable (of interest, for which a hybrid sensor is generated) has a delay between the time of actual sampling and the time a measurement is determined, the following terminology can be used: “collection time epoch” refers to the time 408 when a sample is extracted (e.g., sent for analysis, such as to a lab to determine a measurement); “report time epoch” refers to the time 410 when a lab report (or another analytical report) is available for the corresponding collected sample after a lab-related delay (or another analytical delay).

[0071] To generate training data including input-output data, the method can align or shift lab report values ​​(or measurements of target variables having a delay between the collection time epoch and the report time epoch) arriving at a "report time" relative to a corresponding "collection time". In an embodiment, the method also divides the time horizon into different time periods for the target metric (e.g., measured using intrusive sampling) based on the time from the collection time epoch. The training input-output data can include, for example, a time series of measured upstream data in a first time window, a time series of measured downstream data in a second time window (as input), and a measured target variable having a collection time after the first time window and before the second time window, where the first time window, collection time, and second time window are consecutive in time. A neural network trained and aggregated based on the training data (e.g., as shown in FIG. 10) can be implemented to provide a hybrid sensor capable of measuring the target variable at a requested or given time. In an embodiment, different variables may have different measurement rates.

[0072] For example, for a given time t, the hybrid sensor measurements at time t are calculated based on: upstream sensor values ​​measured over a time window [t-tau_1,t]; downstream sensor values ​​measured over a time window [t,t+tau_2]; and the measured target variable metric (e.g., lab results for the target metric at "report time"). The time used for training is a shifted time that is aligned or shifted to a corresponding acquisition time (where the acquisition time is less than t) relative to the measured target variable metric. In an embodiment, different variables can have different measurement rates. Here, "t" is used to mean the current time, e.g., when the system disclosed herein is being used or is actually being run. Relative to real time "t", the system may use a time (t-tau Dmax ) can be used to generate predictions for the target variable at such time points, i.e., time (t-tau Dmax ), the system is guaranteed to preserve all downstream variables at their respective "lead" times, because the most restrictive such "lead" time is the time when the "lead" time is tau Dmax We use (t-tau Dmax ), the system has access to all measurements of the downstream variables.

[0073] In an embodiment, the method can use a near-continuous-time causal graph, for example using a bidirectional approach using forward influences from upstream variables and backward influences from downstream variables, to provide a prediction of a target variable (e.g., a lab variable). A previously measured target variable can also be used to predict the next target variable. A prediction can be made based on these three sets of measurements, for example using a combined prediction. A prediction can also be made based on these three sets of measurements, for example using a combination of individual measurements predicted for the set of measurements.

[0074] In an embodiment, the method can match a low rate monitoring of a sensor with a high rate monitoring of another sensor, for example, the method can fill gaps in the low rate monitoring with predicted values ​​using both the causality map and the true values.

[0075] In embodiments, the method can generate hybrid sensors to solve problems or difficulties associated with sensors with low monitoring rates. For example, the method can generate hybrid sensors where high frequency and automatic monitoring is difficult or not possible, such as in tailings monitoring, e.g., for these less frequently monitored sensors. In aspects, the method can use low rate measurements in addition to using causal graphs to discover values ​​not directly measured by the sensor.

[0076] In an embodiment, a method, system, and method can be provided that generate hybrid sensors for target variables or metrics that include expensive intrusive measurements with large sampling intervals (low frequency measurements) and delays for analytical (e.g., laboratory) operations in industrial processes, such as in manufacturing and chemical production processes. In an embodiment, the method generates a qualitative causal diagram or data structure showing, for example, links of what are the upstream and downstream causal ends. The method receives sensor data from measured variables in the industrial process at their available rates of sampling and measurement. The method also predicts the target variable at the same or similar high rate as the set of upstream and downstream variables. The method can further generate a hybrid sensor that uses the most recent lab measurements and the predicted values ​​for the target variable. The method can send a signal message, such as, but not limited to, a hazard release alarm, a productivity related process alert or another alert, or both, to the industrial process based on the results of the generated hybrid sensor. The method can also actuate a physical actuator to invoke or control a set point in an industrial process or to correct an error condition determined based on the value of the hybrid sensor.

[0077] In an embodiment, the method can generate a qualitative causal diagram or data structure by, for example: determining a candidate upstream set of variables that have higher rates of sampling and measurement relative to the target variable based on the causal relationship in the process between these candidate upstream set of variables and the target variable; and determining a candidate downstream set of variables that have higher rates of sampling and measurement relative to the target variable based on the causal relationship in the process between these candidate downstream set of variables and the target variable.

[0078] In an embodiment, a combined prediction of the three individual estimates can provide a higher frequency of predictions for the target variable. The combined prediction includes bidirectional prediction. For example, the method uses forward prediction to predict the target variable using a candidate upstream set of variables in a time window specific for this prediction. The method also uses backward prediction to perform a backward in time estimation of the target variable using a candidate downstream set of metric values ​​in a time window specific for this estimation. The method further uses historical values ​​of the true measurements of the target variable. The method can generate a hybrid sensor using aggregate prediction by aggregating the three estimates described above and can determine or predict the value of the target variable (hybrid sensor) at a given time using, for example, weights and attention for each of the three predictions.

[0079] In embodiments, forward and backward predictions can be made using physical or chemical principles to develop empirical inferential models, or using deep learning models based on data science approaches, or both.

[0080] In an embodiment, time windows can be generated for forward and backward predictions by using physical dwell times and physical processing times to calculate time windows and lags of causal relationships between multiple variables, or using time series correlation to analyze time windows using a data-driven approach, or both.

[0081] FIG. 12 is another flow diagram illustrating a method of an embodiment. At 1202, the method includes receiving a first time series of upstream sensor data having a forward dependency on the target variable. The first time series of upstream sensor data is one-dimensional or multi-dimensional time series data. At 1204, the method includes receiving a second time series of downstream sensor data having a backward dependency on the target variable. The second time series of downstream sensor data is one-dimensional or multi-dimensional time series data. At 1206, the method includes receiving a time series of measured target variable data associated with the target variable. The target variable includes a measurement frequency that is lower than a measurement frequency associated with the upstream sensor data and the downstream sensor data. At 1208, the method includes determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable. At 1210, the method includes determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable. At 1212, the method includes training a machine learning model, such as a neural network, based on a training dataset including at least upstream sensor data of a first time series in a first time window, downstream sensor data of a second time series in a second time window, and target variable data of the measured time series to predict a value of the target variable at a given time. The training dataset can include multiple such sets of time series data.

[0082] The method also includes generating a causal data structure that links the upstream sensor data of the first time series forward in time to the measured target variable of the time series and links the downstream sensor data of the second time series backward in time to the measured target variable of the time series. The upstream sensor data and the downstream sensor data can be selected based on the causal data structure.

[0083] In an embodiment, the trained machine learning model, such as a neural network, predicts a value of a target variable at a given time. In an embodiment, the hardware processor further controls a set point in an industrial process based on the predicted value of the target variable for the given time. For example, the industrial process includes oil sands processing and the target variable includes a characteristic associated with a tailings.

[0084] In an embodiment, a machine learning model, such as a neural network, includes an aggregation of a first neural network trained to predict a value of a target variable based on a forward dependency of upstream sensor data of a first time series, a second neural network trained to predict a value of the target variable based on a backward dependency of downstream sensor data of a second time series, and a third neural network trained to predict a value of the target variable based on the measured target variable data of the time series.

[0085] In an embodiment, the report time of the measured target variable has a delay from the time the target variable was collected. A training dataset including time series measured target variable data may include measured target variable data determined at the report time and shifted to the corresponding collection time of the measured target variable. In an embodiment, a machine learning model, such as a neural network, predicts the value of the target variable at a given time without delay at run time.

[0086] FIG. 13 illustrates components of a system in one embodiment that can generate an embodiment hybrid sensor. One or more hardware processors 1302, such as a central processing unit (CPU), a graphic processor unit (GPU), an ater or a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another processor, or a combination thereof, are coupled to a memory device 1304 to generate a hybrid sensor. The memory device 1304 can include random access memory (RAM), read only memory (ROM), or another memory device, and can store data and / or processor instructions for implementing various functions associated with the methods and / or systems described herein. The one or more processors 1302 execute computer instructions stored in the memory 1304 or received from another computer device or medium. The memory device 1304 can store instructions and / or data for the functions of the one or more hardware processors 1302, and can include an operating system and other programs for instructions and / or data. The one or more hardware processors 1302 can receive inputs, for example, a first time series of upstream sensor data having a forward dependency of the target variable, a second time series of downstream sensor data having a backward dependency of the target variable, and a time series of measured target variable data related to the target variable. The upstream sensor data of the first time series can be one-dimensional or multi-dimensional time series of data. The downstream sensor data of the second time series can be one-dimensional or multi-dimensional time series of data. In one aspect, the one or more hardware processors 1302 can also determine a first time window representing a lag of the forward dependency, and a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable.The one or more hardware processors 1302 can train a machine learning model, such as a neural network, based on the training data set described above. The hardware processor 1302 can run the machine learning model, such as a neural network, to predict a value of a target variable at a given time. Based on the predicted value of the target variable for a given time, the hardware processor 1302 can further control a set point in an industrial process. In an embodiment, the training data can be stored in the storage device 1306 or can be received from a remote device via a network interface 1308 and temporarily loaded into the memory device 1304 for training the machine learning model, such as a neural network. The one or more hardware processors 1302 can be coupled to a network interface 1308 for communicating with a remote system over a network, for example, and an input / output interface 1310 for communicating with input or output devices, such as a keyboard, a mouse, a display, or the like, or a combination thereof, or both.

[0087] FIG. 14 illustrates a schematic of an example computer or processing system on which the hybrid sensor generation system in one embodiment may be implemented. The computer system is merely one example of a suitable processing system and is not intended to suggest any limitation as to the scope of the embodiments or functionality of the methodology described herein. The illustrated processing system may be operational with many other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments or configurations or combinations thereof that may be suitable for use with the processing system illustrated in FIG. 14 may include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices.

[0088] The computer system has been described in the general context of computer system executable instructions, such as program modules, executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system may be practiced in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote system storage media, including memory storage devices.

[0089] Components of the computer system include, but are not limited to, one or more processors or processing units 12, a system memory 16, and a bus 14 that couples various components, including the system memory 16, to the processor 12. The processor 12 may include modules 30 that perform the methods described herein. The modules 30 may be programmed within integrated circuits of the processor 12 or may be loaded from the memory 16, the storage device 18, or the network 24 or communication therewith.

[0090] Bus 14 may represent any one or more of several bus structures including a memory bus, memory controller, peripheral bus, graphics accelerated port, and local bus using any of a variety of processor or bus architectures. By way of example, and not limitation, such architectures may include Industrial Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI).

[0091] The computer system may include a variety of computer system readable media, which may be any available media that can be accessed by the computer system and includes both volatile and nonvolatile media, removable and non-removable media.

[0092] System memory 16 may include computer readable media in the form of volatile memory such as random access memory (RAM) or cache memory or both or others. The computer system may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 18 may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (e.g., referred to as a "hard drive"). Although not shown, a removable, non-volatile magnetic disk drive (e.g., a "floppy disk") and an optical disk drive may be provided for reading from and writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical media. In such an example, each may be connected to bus 14 by one or more media interfaces.

[0093] The computer system may also communicate with one or more external devices 26, such as a keyboard, pointing device, display 28, one or more devices that allow a user to interact with the computer system, or any device (e.g., network card, modem, etc.) that allows the computer system to communicate with one or more other computing devices, or a combination thereof. Such communication may occur through an input / output (I / O) interface 20.

[0094] Additionally, the computer system may communicate with one or more networks 24, such as a local area network (LAN), a general purpose wide area network (WAN), or a public network (the Internet), or combinations thereof, via a network adapter 22. As shown, the network adapter 22 communicates with other components of the computer system via a bus 14. It should be understood that other hardware and / or software components, not shown, could be used in combination with the computer system. Examples include, but are not limited to, microcode, device drivers, redundant processing units, and external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0095] The present invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product includes a computer-readable recording medium (or media) having computer-readable program instructions thereon for causing a processor to perform features of the present disclosure.

[0096] A computer readable medium may be any tangible device capable of holding and storing a plurality of instructions for use by an instruction execution device. A computer readable medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electro-magnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards, or mechanically encoded devices having protruding structures within grooves that record the instructions, and any suitable combination thereof. As used herein, a computer-readable recording medium is not to be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves such as wave guides or other communications media (e.g., light pulses passing through fiber optic cable), or electrical signals communicated through wires.

[0097] The computer programs described herein can be downloaded from a computer readable recording medium into each computing / processing device, or can be downloaded to an external computer or external recording device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, and combinations thereof. The network can include copper cables, fiber optics, wireless networks, routers, firewalls, switches, gateway computers, and edge servers, and combinations thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions into a computer readable recording medium in each computing / processing device for storage.

[0098] The computer readable program instructions for carrying out the operations of the present invention can be either source code or object code written in any combination of programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or one or more procedural programming languages, such as Smalltalk®, object oriented programming languages ​​such as C++, the “C” programming language, or similar programming languages. The computer readable program instructions can be executed entirely on the user computer, partially on the user computer as a stand-alone software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user computer through any type of network, including a local area network (LAN), a wide area network (WAN), or the connection can be to an external computer (e.g., through an Internet service provider). In some embodiments, computer readable program instructions may be executed by electrical circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), and state information of the computer readable program instructions may be used to personalize the electrical circuitry to perform features of the present invention.

[0099] Aspects of the invention described herein have been described with reference to flowchart instructions and / or block diagrams of methods, apparatus (systems), and computer-readable storage media and computer program products according to embodiments of the invention. It will be understood that any combination of flowchart illustrations and / or block diagrams and / or blocks in flowchart illustrations and / or block diagrams can be implemented by computer-readable program instructions.

[0100] These computer readable program instructions can be provided to a general purpose computer, a special purpose computer, or other processor or other programmable data processing device to produce a machine, which when executed by the processor of the computer or other programmable data processing device produces means for implementing the functions / operations specified in the block or blocks of the flowcharts and block diagrams, or combinations thereof. These computer readable program instructions that direct the computer, programmable data processing device, and other devices, or combinations thereof, to function in a particular manner can also be stored on a computer readable recording medium, and the computer readable recording medium having instructions stored therein constitutes an article of manufacture containing instructions that implement the functional / operational features specified in the block or blocks of the flowcharts and block diagrams, or combinations thereof.

[0101] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device and cause a computer implemented process to perform a series of operational steps on the computer, other programmable apparatus, or other device to implement the functions / acts specified in the blocks of the flowcharts and block diagrams or in a combination thereof.

[0102] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and possible implementation operations of systems, methods, and computer program products according to various embodiments of the present invention. In this respect, the flowcharts or block diagrams may represent modules, segments, or portions of instructions, which include one or more executable instructions for implementing a particular logical function(s) (or functions). In some alternative implementations, the functions described in the blocks may be performed other than as illustrated. For example, two blocks shown in succession may be actually performed in one step, simultaneously, substantially simultaneously, in a partially or fully overlapped manner in time, or the blocks may sometimes be performed in a reverse order, depending on the functions involved. It is also noted that the illustration of the blocks in the block diagrams and / or the illustration of the flowcharts or combinations thereof may be implemented by a system based on special purpose hardware that performs a particular function or operation or executes specific purpose hardware and computer instructions.

[0103] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "or" is an inclusive operator and can mean "and / or" unless the context explicitly or clearly indicates otherwise. Furthermore, when the terms "comprise", "comprising", or both are used herein, it should be understood that they specify the presence of the stated features, integers, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups, or combinations thereof. As used herein, the phrase "in an embodiment" does not necessarily refer to the same embodiment, although it may. As used herein, the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may. As used herein, the phrase "another embodiment" does not necessarily refer to the same embodiment, although it may. Additionally, embodiments or components of embodiments may be freely combined with each other unless they are mutually exclusive.

[0104] Corresponding structures, materials, acts, and all means or step-plus-function equivalents are intended to include any structure, material, or act for performing the function of a combination with other claimed elements as specifically claimed in the following claims, if any. The description of the present invention has been presented for purposes of illustration and description, but it is not intended to be exhaustive or limited to the invention in the disclosed form. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the invention. The embodiments have been selected and described in order to best explain the principles of the invention, practical application, and embodiments disclosed herein, and to enable others skilled in the art to understand the invention in various embodiments with various modifications that make them suitable for the specific uses envisioned.

Claims

1. 1. A system for creating a hybrid sensor in an industrial process, comprising: The system comprises at least receiving a first time series of upstream sensor data having a forward dependency on a target variable; receiving a second time series of downstream sensor data having a backward dependency on the target variable; receiving a time series of measured target variable data associated with the target variable, the target variable having a measurement frequency that is less than a measurement frequency associated with the upstream sensor data and the downstream sensor data; determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable; determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable; training a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and time-series measured target variable data to predict a value of the target variable at a given time; The system is configured as follows.

2. 2. The system of claim 1, further configured to generate a causal data structure that links the upstream sensor data of the first time series forward in time to the measured target variable data of the time series and links the downstream sensor data of the second time series backward in time to the measured target variable data of the time series.

3. The system of claim 2 , wherein the upstream sensor data and the downstream sensor data are selected based on the causal relationship data structure.

4. 2. The system of claim 1, wherein the trained machine learning model predicts the value of the target variable at the given time, and wherein the system further controls a set point in an industrial process based on the predicted value of the target variable for the given time.

5. 5. The system of claim 4, wherein the industrial process comprises oil sands processing and the target variables comprise properties associated with tailings.

6. 2. The system of claim 1 , wherein the machine learning model comprises an aggregation of a first neural network trained to predict the value of the target variable based on a forward prediction of the upstream sensor data of the first time series, a second neural network trained to predict the value of the target variable based on a backward prediction of the downstream sensor data of the second time series, and a third neural network trained to predict the value of the target variable based on the time series of measured target variable data.

7. 7. The system of claim 6, wherein a report time of a measured target variable has a delay from a collection time of the target variable, and the training dataset including the time series of measured target variables includes data for the measured target variable determined at a report time and shifted to a corresponding collection time of the measured target variable, and the machine learning model predicts the value of the target variable at the given time without the delay at run-time.

8. The system of claim 1 , wherein the first time series of upstream sensor data comprises a multi-dimensional time series of data, and the second time series of downstream sensor data comprises a multi-dimensional time series of data.

9. The system of claim 1 , wherein the machine learning model comprises a neural network model.

10. 1. A computer program for generating a hybrid sensor in an industrial process, the computer program comprising: receiving a first time series of upstream sensor data having a forward dependency on a target variable; receiving a second time series of downstream sensor data having a backward dependency on the target variable; receiving a time series of measured target variable data associated with the target variable, the target variable having a measurement frequency that is less than a measurement frequency associated with the upstream sensor data and the downstream sensor data; determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable; determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable; 1. A computer program product comprising: a computer program product for training a machine learning model based on a training dataset comprising at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and time-series measured target variable data to predict a value of the target variable at a given time.

11. 11. The computer program product of claim 10, further causing the device to generate a causal data structure that links the upstream sensor data of the first time series forward in time to the measured target variable data of the time series and links the downstream sensor data of the second time series backward in time to the measured target variable data of the time series.

12. The computer program product of claim 11 , wherein the upstream sensor data and the downstream sensor data are selected based on the causal relationship data structure.

13. 11. The computer program product of claim 10, wherein the trained machine learning model predicts the value of the target variable at the given time, and wherein the device further controls a set point in an industrial process based on the predicted value of the target variable for the given time.

14. 14. The computer program product of claim 13, wherein the industrial process comprises oil sands processing and the target variables comprise properties associated with tailings.

15. 11. The computer program product of claim 10, wherein the machine learning model comprises an aggregation of a first neural network trained to predict the value of the target variable based on a forward prediction of the upstream sensor data of the first time series, a second neural network trained to predict the value of the target variable based on a backward prediction of the downstream sensor data of the second time series, and a third neural network trained to predict the value of the target variable based on the time series of measured target variable data.

16. 16. The computer program product of claim 15, wherein a report time of a measured target variable has a delay from a collection time of the target variable, and the training dataset including the time series of measured target variables includes data for the measured target variable determined at a report time and shifted to a corresponding collection time of the measured target variable, and the machine learning model predicts the value of the target variable at the given time without the delay at run-time.

17. 11. The computer program product of claim 10, wherein the first time series of upstream sensor data comprises multi-dimensional time series data, and the second time series of downstream sensor data comprises multi-dimensional time series data.

18. The computer program product of claim 10 , wherein the machine learning model comprises a neural network model.

19. 1. A computer-implemented method for generating a hybrid sensor in an industrial process, comprising: receiving a first time series of upstream sensor data having a forward dependency on a target variable; receiving a second time series of downstream sensor data having a backward dependency on the target variable; receiving a time series of measured target variable data associated with the target variable, the target variable having a measurement frequency that is less than a measurement frequency associated with the upstream sensor data and the downstream sensor data; determining a first time window representing a lag of the forward dependency between the upstream sensor data and the target variable; determining a second time window representing a lag of the backward dependency between the downstream sensor data and the target variable; training a machine learning model based on a training dataset including at least upstream sensor data of the first time series within the first time window, downstream sensor data of the second time series within the second time window, and time-series measured target variable data to predict a value of the target variable at a given time; A method comprising:

20. 20. The method of claim 19, wherein the first time series of upstream sensor data comprises a multi-dimensional time series of data, the second time series of downstream sensor data comprises a multi-dimensional time series of data, and the machine learning model comprises a neural network model.

Images