Systems, methods, and computer readable media for dynamic process monitoring and / or generation of master predictor models

Abstract

A method for generating a master predictor model from multi-dimensional time series data includes: receiving, from a plurality of sensors, multi-dimensional time series data corresponding to a plurality of raw variables; transforming the multi-dimensional time series data to a lower dimension to define dimensionality-reduced time series data; extracting, by a controller, a plurality of potential variables from the dimensionality-reduced time series data and determining values of the plurality of potential variables within a first time period; initializing, by the controller, a load matrix corresponding to a set of potential variables of the plurality of potential variables; and determining, by the controller, one or more master predictor model parameters by performing an iterative process. The iterative process includes: (a) predicting values of the plurality of potential variables by using an estimation process that maximizes a covariance between the values of the plurality of potential variables and predicted values of the plurality of potential variables based on the dimensionality reduced time series data and the load matrix; (b) calculating a new load matrix according to the load matrix and the predicted value of the potential variable; (c) updating the load matrix based on the calculated new load matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters reach convergence.

Description

Technical Field

[0001] The embodiments generally relate to systems, methods, and computer-readable media for dynamic process monitoring and fault detection in high-dimensional time series data, and in some embodiments for generating master predictor models from multi-dimensional time series data. Background Technology

[0002] The application of the Industrial Internet of Things (IIoT) in intelligent and autonomous operation generates a wealth of multidimensional time-series data, offering tremendous opportunities for data analysis and information extraction. One such opportunity lies in using massive amounts of data for monitoring, prediction, and decision-making. On the other hand, existing industrial processes have already collected vast amounts of high-dimensional operational data, which can be used for multivariate monitoring and / or fault detection. Opportunities for data-driven monitoring applications include all electrical, mechanical, and civil engineering systems requiring preventative maintenance. Examples include: i) safety monitoring in municipal infrastructure, transportation systems, and utilities; ii) building heating and cooling systems; and iii) industrial manufacturing processes that record multivariate time-series data for quality and safety monitoring.

[0003] Multivariate time data series are sequences of two or more variables that change over time. Two common properties of such multivariate time data series are: i) high-dimensional collinearity, which produces a subspace of cross-correlated changes; and ii) time-varying sequence dependencies exhibiting autocorrelation, which allow subsequent samples to be predicted from earlier samples. Most existing data analysis methods address collinearity issues for process monitoring, but sequence dependencies of data have not been adequately studied except in model-based fault detection systems. In recent years, the combined problems of data collinearity and dynamics have been systematically considered as potential frameworks for dimensionality-reduced dynamic modeling. Dynamic intrinsic principal component analysis (DiPCA) and dynamic intrinsic canonical correlation analysis (DiCCA) methods, which are considered as univariate latent dynamics, have been developed using univariate latent autoregressive (AR) models. Latent variables (also called 'hidden variables') are variables that are not directly observed, nor are they one of the observed variables, but are inferred from one or more of the observed variables. Latent variables represent underlying characteristics, factors, or constructs that cannot be directly measured but can be estimated through their relationship with observed data. An autoregressive model is a model that predicts future values ​​based at least on past values. Univariate latent autoregressive models are autoregressive models based on a single latent variable (rather than multiple latent variables). DiPCA and DiCCA methods extract one dynamic latent variable (DLV) at a time, making that dynamic latent variable most predictable from its past data. Therefore, each of the DLVs is modeled as a univariate autoregressive (AR) process. These univariate AR models can be interpreted as marginal models that decouple at the cost of increased variance in model error. A significant drawback of these non-interactive AR models is that they can be inefficient in extracting DLV dynamics when the true latent structures interact. This inefficiency often leads to the use of very high-order AR models and an excessive number of DLVs. To address this problem, iterative algorithms have been developed that employ vector AR models for all latent variables. However, these iterative algorithms are sequential, one DLV after another. When the algorithm iterates from the (l–1)th DLV to the lth DLV, the weight matrix of the previously (l–1)th DLV is not updated, which is suboptimal.

[0004] To include interactive latent dynamic relationships, attempts have been made to use vector AR latent dynamic models and / or project vector AR latent models onto a probabilistic framework for parameter estimation. However, this work uses canonical correlation as the objective function, which ignores the variance magnitude of the extracted dynamic latent variables.

[0005] Overall, despite significant progress in modeling and monitoring dynamic latent variables, challenges remain in effectively capturing and predicting the variance of high-dimensional time series data.

[0006] Any discussion of documents, actions, materials, equipment, articles, etc., included in this specification shall not be construed as an admission that any or all of these matters constitute part of the prior art or, due to their existence prior to the priority date of the claims of this application, form common general knowledge in the field relating to this disclosure. Summary of the Invention

[0007] Some embodiments relate to a method for generating a master predictor model from multidimensional time series data, the method comprising: receiving multidimensional time series data corresponding to multiple original variables from multiple sensors; transforming the multidimensional time series data to a lower dimension to define dimensionality-reduced time series data; extracting multiple latent variables from the dimensionality-reduced time series data by a controller and determining the values ​​of the multiple latent variables within a first time period; initializing a loading matrix corresponding to a set of latent variables among the multiple latent variables by the controller; and determining one or more master predictor model parameters by the controller through an iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the dimensionality-reduced time series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0008] In some embodiments, transforming the multidimensional time series data to a lower dimension may include performing singular value decomposition (SVD) on the multidimensional time series data.

[0009] The method may further include generating a latent vector autoregressive (LaVAR) model based on one or more determined master predictor model parameters. The master predictor model may include a maximum prediction variance (MPV) objective.

[0010] In some embodiments, the estimation process can simultaneously maximize the covariance of the predicted values ​​of the plurality of latent variables. The estimation process can simultaneously maximize the covariance of the predicted variability among the predicted values ​​of the plurality of latent variables.

[0011] Calculating the new loading matrix may include performing eigenvalue decomposition on the matrix of predicted values ​​of the plurality of latent variables to compute the new loading matrix.

[0012] In some embodiments, the method may further include determining the number of latent variables in the set of latent variables. Determining the number of latent variables allows the latent variables to capture a target predictable amount of change in the dimensionality-reduced time series data. The determination of the number of latent variables may be based on a target proportion of predicted variance (PPV). The number of latent variables may be determined such that the target PPV represents a majority of the total predicted variance. The number of latent variables may be determined such that the target PPV represents at least 95% of the total predicted variance.

[0013] The method may further include determining the orthogonal complement of the load matrix to define a static load matrix.

[0014] In some embodiments, the method may further include: analyzing the multidimensional time series data corresponding to the dynamic system using the master predictor model; generating one or more monitoring indicators of the predicted residuals and / or latent variables of the analyzed multidimensional time series data; and detecting anomalies in the multidimensional time series data based on the one or more monitoring indicators.

[0015] Generating one or more monitoring indicators may include: a) generating one or more first monitoring indicators of the prediction residuals; b) generating one or more second monitoring indicators of the latent variables; and c) generating one or more combined monitoring indicators based on the one or more first monitoring indicators of the prediction residuals and the one or more second monitoring indicators of the latent variables.

[0016] The one or more first monitoring indicators of the predicted residuals may include at least one of a first Hotelling index and a squared prediction error (SPE) index defined for the predicted residuals. The one or more first monitoring indicators of the predicted residuals may include a combination of the first Hotelling index and the SPE index. The one or more second monitoring indicators of the latent variable may include a second Hotelling index defined for the predicted latent variable.

[0017] In some embodiments, the method may further include generating one or more overall monitoring metrics for both the prediction residuals and the prediction latent variables by: determining a first Hotelling metric for the prediction residuals; determining a squared prediction error (SPE) metric for the prediction residuals; combining the first Hotelling metric and the SPE metric for the prediction residuals to generate one or more combined monitoring metrics for the prediction residuals; determining a second Hotelling metric for the prediction latent variables; and combining the second Hotelling metric for the prediction latent variables with the one or more combined monitoring metrics for the prediction residuals.

[0018] Some embodiments relate to a method for identifying anomalies detected in multidimensional time series data using a master predictor model generated by the method disclosed herein, the method comprising: determining a prediction error matrix based on the difference between predicted values ​​and actual values ​​of the multidimensional time series data; decomposing the prediction error matrix to identify anomaly directions within the prediction error matrix; determining a reconstruction-based contribution (RBC) matrix, the reconstruction-based contribution matrix including the contribution of each variable to the detected anomaly; and analyzing the RBC matrix to identify the detected anomaly.

[0019] In some embodiments, the anomaly direction can be used to reconstruct anomaly-free data. In some embodiments, reconstructing the anomaly-free data may include: projecting the prediction error matrix onto the anomaly direction, and subtracting the contribution of the detected anomalies from the multidimensional time series data.

[0020] Some embodiments relate to a non-transitory computer-readable storage medium storing instructions that, when executed by a processing device, cause the processing device to perform any of the methods disclosed herein.

[0021] Some embodiments relate to a computing device that includes a non-transitory computer-readable storage medium disclosed herein and a processor configured to execute instructions stored in the non-transitory computer-readable storage medium.

[0022] Some embodiments relate to a system for generating a master predictor model from multidimensional time series data, the system comprising: a plurality of sensors configured to detect multidimensional time series data corresponding to a plurality of original variables; and a processing unit coupled to the plurality of sensors, the processing unit being configured to perform any of the methods disclosed herein.

[0023] Some embodiments provide a system for generating a master predictor model from multidimensional time series data, the system comprising: a plurality of sensors configured to detect multidimensional time series data corresponding to a plurality of original variables; and a processing unit coupled to the plurality of sensors, the processing unit being configured to: transform the multidimensional time series data to a lower dimension to define dimensionality-reduced time series data; extract a plurality of latent variables from the dimensionality-reduced time series data by a controller and determine the values ​​of the plurality of latent variables within a first time period; initialize a loading matrix corresponding to a set of latent variables among the plurality of latent variables by the controller; and by... The controller determines one or more master predictor model parameters by performing an iterative process comprising: (a) predicting the values ​​of the plurality of latent variables based on the dimensionality-reduced time series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the plurality of latent variables and the predicted values ​​of the plurality of latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0024] Some embodiments provide a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: receiving multidimensional time-series data corresponding to multiple original variables from multiple sensors; transforming the multidimensional time-series data to a lower dimension to define dimensionality-reduced time-series data; extracting multiple latent variables from the dimensionality-reduced time-series data and determining the values ​​of the multiple latent variables within a first time period; initializing a loading matrix corresponding to a set of latent variables among the multiple latent variables; and determining one or more master predictor model parameters by performing an iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the dimensionality-reduced time-series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0025] Throughout this specification, the word “comprise” or variations such as “comprises / comprising” will be understood to imply inclusion of the stated elements, integers or steps, or groups of elements, integers or steps, but does not exclude any other elements, integers or steps, or groups of elements, integers or steps. Attached Figure Description

[0026] The figures in the accompanying drawings are merely example embodiments of this disclosure and should not be construed as limiting the scope of this disclosure.

[0027] Figure 1 This is a process flowchart of a method for generating a master predictor model from multidimensional time series data according to some embodiments;

[0028] Figure 2A This is a process flowchart of a method for detecting anomalies in a dynamic system according to some embodiments;

[0029] Figure 2B This is a process flowchart of a method for generating one or more monitoring indicators according to some embodiments;

[0030] Figure 3A This is a process flowchart of a method for generating one or more overall monitoring indicators for predicting residuals and predicting latent variables, according to some embodiments;

[0031] Figure 3B This is a process flowchart of a PPA-based method for monitoring and detecting faults according to some embodiments;

[0032] Figure 4 These are diagrams illustrating examples of dynamic systems with process failures according to some embodiments;

[0033] Figure 5A It is aimed at Figure 4 The example system in the example uses DLV's φ e and φ o A comparison of monitoring charts based on DiCCA and PCA with monitoring charts based on DiCCA and PCA.

[0034] Figure 5B It is aimed at Figure 4 The example system in the example uses DLV's φ e and φ o Another illustration comparing the monitoring charts based on DiCCA and PCA with the monitoring charts based on DiCCA and PCA;

[0035] Figure 6 This is a table showing 23 measurement variables studied in 19 distributed control loops according to some embodiments;

[0036] Figure 7 The graphs showing the CPV(l) of LaVAR-MPV, LaVAR-CCA and DiCCA according to some embodiments are shown.

[0037] Figure 8A The following diagram illustrates the use of monitoring metrics φ for the LaVAR-MPV model, as well as DiCCA and PCA, according to some embodiments. e φ o φ eDiCCA φ oDiCCA Q y and φ y Fault detection rate (FDR) and false alarm rate (FAR);

[0038] Figure 8B The following are examples illustrating the use of monitoring metrics φ for the PPA model and DiCCA and PCA, according to some embodiments. e φ o φ eDiCCA φ oDiCCA Q yPCA and φ yPCA Another illustration of the fault detection rate (FDR) and false alarm rate (FAR);

[0039] Figure 9A This refers to the use of 8 DLVs for IDV2 and the LaVAR-MPV model according to some embodiments, φ e and φ o A comparison of monitoring charts based on DiCCA and PCA with monitoring charts based on DiCCA and PCA.

[0040] Figure 9B This refers to the use of 8 DLVs for IDV2 and the φ of the PPA model according to some embodiments. e and φ o Another illustration comparing the monitoring charts based on DiCCA and PCA with the monitoring charts based on DiCCA and PCA;

[0041] Figure 10 This refers to the use of 8 DLVs for IDV11 and the LaVAR-MPV model according to some embodiments, φ e and φ o A comparison of monitoring charts based on DiCCA and PCA with those based on PCA; and

[0042] Figure 11 This refers to the use of 8 DLVs for IDV13 and the LaVAR-MPV model according to some embodiments. e and φ o A comparison of monitoring charts based on DiCCA and PCA with monitoring charts based on DiCCA and PCA.

[0043] Figure 12 The graphs show a comparison of the total variance proportion (PTV) (top) and the predicted variance proportion (PPV) (bottom) of the PPA, LaVAR, and DiCCA models on the test dataset.

[0044] Figure 13 This shows the monitoring index φ of the PPA model for interference 1 and interference 2. e A curve graph;

[0045] Figure 14 It shows a graph of training and test data up to June 15, separated by a vertical dashed line;

[0046] Figure 15 It is a statistical chart showing the contribution of each variable to interference 1 and interference 2 respectively;

[0047] Figure 16 The monitoring indicators for interference 1 and interference 2 are shown. and φ e A curve graph;

[0048] Figure 17 This is a set of graphs showing the raw values ​​and PPA-generated values ​​for anomalous periods in the training and test datasets; and

[0049] Figure 18 This is a block diagram of a system configured to generate one or more master predictor models according to some embodiments. Implementation Example Description Numbered Examples – Group 1

[0050] Example 1.1. A method for generating a master predictor model from multidimensional time series data, the method comprising: receiving multidimensional time series data corresponding to multiple original variables from multiple sensors; transforming the multidimensional time series data to a lower dimension to define dimensionality-reduced time series data; extracting multiple latent variables from the dimensionality-reduced time series data by a controller and determining the values ​​of the multiple latent variables in a first time period; initializing a loading matrix corresponding to a set of latent variables among the multiple latent variables by the controller; determining one or more master predictor model parameters by the controller through an iterative process, the iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the dimensionality-reduced time series data and the loading matrix by using an estimation process, the estimation process maximizing the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0051] Example 1.2. According to the method described in Example 1.1, transforming the multidimensional time series data to a lower dimension includes performing singular value decomposition (SVD) on the multidimensional time series data.

[0052] Example 1.3. The method according to any one of Examples 1.1 to 1.2 further includes generating a latent vector autoregressive (LaVAR) model based on one or more determined master predictor model parameters.

[0053] Example 1.4. The method according to any one of Examples 1.1 to 1.3, wherein the master predictor model includes a maximum prediction variance (MPV) objective.

[0054] Example 1.5. The method according to any one of Examples 1.1 to 1.4, wherein the estimation process simultaneously maximizes the covariance of the predicted values ​​of the plurality of latent variables.

[0055] Example 1.6. The method according to Example 1.5, wherein the estimation process simultaneously maximizes the covariance of the predicted variability in the predicted values ​​of the plurality of latent variables.

[0056] Example 1.7. The method according to any one of Examples 1.1 to 1.6, wherein calculating the new load matrix includes performing eigenvalue decomposition on the matrix of predicted values ​​of the plurality of latent variables to calculate the new load matrix.

[0057] Example 1.8. The method according to any one of Examples 1.1 to 1.7 further includes determining the number of latent variables in the set of latent variables.

[0058] Example 1.9. According to the method of Example 1.8, wherein the number of latent variables is determined to capture the target predictable change in the dimensionality-reduced time series data through the latent variables.

[0059] Example 1.10. The method according to any one of Examples 1.8 to 1.9, wherein the determination of the number of latent variables is based on the target prediction variance ratio (PPV).

[0060] Example 1.11. The method according to Example 1.10, wherein the number of latent variables is determined such that the target PPV accounts for the majority of the total prediction variance.

[0061] Example 1.12. The method according to any one of Examples 1.10 to 1.11, wherein the number of latent variables is determined such that the target PPV accounts for at least 95% of the total prediction variance.

[0062] Example 1.13. The method according to any one of Examples 1.1 to 1.12 further includes determining the orthogonal complement of the load matrix to define the static load matrix.

[0063] Example 1.14. The method for generating a master predictor model from multidimensional time series data according to any one of Examples 1.1 to 1.13 further includes: analyzing the multidimensional time series data corresponding to a dynamic system using the master predictor model; generating one or more monitoring indicators of the prediction residuals and / or latent variables of the analyzed multidimensional time series data; and detecting anomalies in the multidimensional time series data based on the one or more monitoring indicators.

[0064] Example 1.15. The method according to any one of Examples 1.1 to 1.14, wherein generating one or more monitoring indicators includes: a) generating one or more first monitoring indicators of the prediction residual; b) generating one or more second monitoring indicators of the latent variable; and c) generating one or more combined monitoring indicators based on the one or more first monitoring indicators of the prediction residual and the one or more second monitoring indicators of the latent variable.

[0065] Example 1.16. The method according to Example 1.15, wherein the one or more first monitoring indicators of the prediction residual include at least one of a first Hotelling index and a squared prediction error (SPE) index defined for the prediction residual.

[0066] Example 1.17. The method according to any one of Examples 1.15 to 1.16, wherein the one or more first monitoring indicators of the predicted residuals include a combination of the first Hotelling indicator and the SPE indicator.

[0067] Example 1.18. The method according to any one of Examples 1.15 to 1.17, wherein the one or more second monitoring indicators of the latent variable include a second Hotelling indicator defined for the predictive latent variable.

[0068] Example 1.19. The method according to any one of Examples 1.1 to 1.18, wherein the method further comprises generating one or more overall monitoring indicators for both the prediction residuals and the prediction latent variables by: determining a first Hotelling index for the prediction residuals; determining a squared prediction error (SPE) index for the prediction residuals; combining the first Hotelling index and the SPE index of the prediction residuals to generate one or more combined monitoring indicators for the prediction residuals; determining a second Hotelling index for the prediction latent variables; and combining the second Hotelling index of the prediction latent variables with the one or more combined monitoring indicators for the prediction residuals.

[0069] Example 1.20. A system for generating a master predictor model from multidimensional time series data, the system comprising: a plurality of sensors configured to detect multidimensional time series data corresponding to a plurality of original variables; and a processing unit coupled to the plurality of sensors, the processing unit being configured to: transform the multidimensional time series data to a lower dimension to define dimensionality-reduced time series data; extract a plurality of latent variables from the dimensionality-reduced time series data by a controller and determine the values ​​of the plurality of latent variables within a first time period; initialize a loading matrix corresponding to a set of latent variables among the plurality of latent variables by the controller; and by... The controller determines one or more master predictor model parameters by performing an iterative process comprising: (a) predicting the values ​​of the plurality of latent variables based on the dimensionality-reduced time series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the plurality of latent variables and the predicted values ​​of the plurality of latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0070] Example 1.21. The system according to Example 1.20, wherein extracting multiple latent variables from the multidimensional time series data and transforming the multidimensional time series data to a lower dimension includes performing singular value decomposition (SVD) on the multidimensional time series data.

[0071] Example 1.22. The system according to any one of Examples 1.20 to 1.21, wherein the processing unit is further configured to generate a latent vector autoregressive (LaVAR) model based on one or more determined master predictor model parameters.

[0072] Example 1.23. The system according to any one of Examples 1.20 to 1.22, wherein the master predictor model includes a maximum prediction variance (MPV) objective.

[0073] Example 1.24. The system according to any one of Examples 1.20 to 1.23, wherein the estimation process simultaneously maximizes the covariance of the predicted values ​​of the plurality of latent variables.

[0074] Example 1.25. The system according to Example 1.24, wherein the estimation process simultaneously maximizes the covariance of the predicted variability in the predicted values ​​of the plurality of latent variables.

[0075] Example 1.26. The system according to any one of Examples 1.20 to 1.25, wherein calculating the new load matrix includes performing eigenvalue decomposition on the matrix of predicted values ​​of the plurality of latent variables to calculate the new load matrix.

[0076] Example 1.27. The system according to any one of Examples 1.20 to 1.26, wherein the processing unit is further configured to determine the number of latent variables in the set of latent variables.

[0077] Example 1.28. The system according to Example 1.27, wherein determining the number of latent variables captures the target predictable changes in the dimensionality-reduced time series data through the latent variables.

[0078] Example 1.29. The system according to any one of Examples 1.27 to 1.28, wherein the determination of the number of latent variables is based on the target prediction variance ratio (PPV).

[0079] Example 1.30. The system according to Example 1.29, wherein the number of latent variables is determined such that the target PPV accounts for the majority of the total prediction variance.

[0080] Example 1.31. The system according to any one of Examples 1.29 to 1.30, wherein the number of latent variables is determined such that the target PPV accounts for at least 95% of the total prediction variance.

[0081] Example 1.32. The system according to any one of Examples 1.20 to 1.31, wherein the processing unit is further configured to determine the orthogonal complement of the load matrix to define a static load matrix.

[0082] Example 1.33. The system according to any one of Examples 1.20 to 1.32, wherein the processing unit is further configured to: analyze the multidimensional time series data corresponding to the dynamic system using the master predictor model; generate one or more monitoring indicators of the predicted residuals and / or latent variables of the analyzed multidimensional time series data; and detect anomalies in the multidimensional time series data based on the one or more monitoring indicators.

[0083] Example 1.34. The system according to any one of Examples 1.20 to 1.33, wherein generating one or more monitoring indicators includes: a) generating one or more first monitoring indicators of the prediction residual; b) generating one or more second monitoring indicators of the latent variable; and c) generating one or more combined monitoring indicators based on the one or more first monitoring indicators of the prediction residual and the one or more second monitoring indicators of the latent variable.

[0084] Example 1.35. The system according to Example 1.34, wherein the one or more first monitoring indicators of the prediction residual include at least one of a first Hotelling index and a squared prediction error (SPE) index defined for the prediction residual.

[0085] Example 1.36. The system according to any one of Examples 1.34 to 1.35, wherein the one or more first monitoring indicators of the predicted residuals include a combination of the first Hotelling indicator and the SPE indicator.

[0086] Example 1.37. The system according to any one of Examples 1.34 to 1.36, wherein the one or more second monitoring indicators of the latent variable include a second Hotelling indicator defined for the predictive latent variable.

[0087] Example 1.38. The system according to any one of Examples 1.20 to 1.37, wherein the processing unit is further configured to generate one or more overall monitoring indicators for both the prediction residuals and the prediction latent variables by: determining a first Hotelling index for the prediction residuals; determining a squared prediction error (SPE) index for the prediction residuals; combining the first Hotelling index and the SPE index of the prediction residuals to generate one or more combined monitoring indicators for the prediction residuals; determining a second Hotelling index for the prediction latent variables; and combining the second Hotelling index of the prediction latent variables with the one or more combined monitoring indicators for the prediction residuals.

[0088] Example 1.39. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: receiving multidimensional time-series data corresponding to multiple original variables from multiple sensors; transforming the multidimensional time-series data to a lower dimension to define dimensionality-reduced time-series data; extracting multiple latent variables from the dimensionality-reduced time-series data and determining the values ​​of the multiple latent variables within a first time period; initializing a loading matrix corresponding to a set of latent variables among the multiple latent variables; determining one or more master predictor model parameters by performing an iterative process, the iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the dimensionality-reduced time-series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0089] Example 1.40. The non-transitory computer-readable medium according to Example 1.39, wherein transforming the multidimensional time series data to a lower dimension includes performing singular value decomposition (SVD) on the multidimensional time series data.

[0090] Example 1.41. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.40, wherein the one or more processors generate a latent vector autoregressive (LaVAR) model based on one or more determined master predictor model parameters.

[0091] Example 1.42. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.41, wherein the master predictor model includes a maximum prediction variance (MPV) objective.

[0092] Example 1.43. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.42, wherein the estimation process simultaneously maximizes the covariance of the predicted values ​​of the plurality of latent variables.

[0093] Example 1.44. The non-transitory computer-readable medium according to Example 1.43, wherein the estimation process simultaneously maximizes the covariance of the predicted variability in the predicted values ​​of the plurality of latent variables.

[0094] Example 1.45. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.44, wherein computing a new load matrix comprises performing eigenvalue decomposition on a matrix of predicted values ​​of the plurality of latent variables to compute the new load matrix.

[0095] Example 1.46. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.45, wherein the one or more processors are made to determine the number of latent variables in the set of latent variables.

[0096] Example 1.47. A non-transitory computer-readable medium according to Example 1.46, wherein determining the number of latent variables captures the target predictable changes in the dimensionality-reduced time series data through the latent variables.

[0097] Example 1.48. A non-transitory computer-readable medium according to any one of Examples 1.46 to 1.47, wherein the determination of the number of latent variables is based on the target prediction variance ratio (PPV).

[0098] Example 1.49. A non-transitory computer-readable medium according to Example 1.48, wherein the number of latent variables is determined such that the target PPV accounts for a majority of the total prediction variance.

[0099] Example 1.50. A non-transitory computer-readable medium according to any one of Examples 1.48 to 1.49, wherein the number of latent variables is determined such that the target PPV accounts for at least 95% of the total prediction variance.

[0100] Example 1.51. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.50, wherein the one or more processors determine the orthogonal complement of the load matrix to define a static load matrix.

[0101] Example 1.52. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.51, wherein the one or more processors are caused to perform the following operations: analyze dimensionality-reduced time series data corresponding to a dynamic system using the master predictor model; generate one or more monitoring indicators of the predicted residuals and / or latent variables of the analyzed dimensionality-reduced time series data; and detect anomalies in the dimensionality-reduced time series data based on the one or more monitoring indicators.

[0102] Example 1.53. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.52, wherein generating one or more monitoring indicators comprises: a) generating one or more first monitoring indicators of the prediction residual; b) generating one or more second monitoring indicators of the latent variable; and c) generating one or more combined monitoring indicators based on the one or more first monitoring indicators of the prediction residual and the one or more second monitoring indicators of the latent variable.

[0103] Example 1.54. The non-transitory computer-readable medium according to Example 1.53, wherein the one or more first monitoring indicators of the prediction residual include at least one of a first Hotelling index and a squared prediction error (SPE) index defined for the prediction residual.

[0104] Example 1.55. A non-transitory computer-readable medium according to any one of Examples 1.53 to 1.54, wherein the one or more first monitoring indicators of the predicted residuals include a combination of the first Hotelling indicator and the SPE indicator.

[0105] Example 1.56. A non-transitory computer-readable medium according to any one of Examples 1.53 to 1.55, wherein the one or more second monitoring indicators of the latent variable include a second Hotelling index defined for the predictive latent variable.

[0106] Example 1.57. A non-transitory computer-readable medium according to any one of Examples 1.39 to 1.56, wherein the one or more processors generate one or more overall monitoring metrics for both the prediction residuals and the prediction latent variables by: determining a first Hotelling metric for the prediction residuals; determining a squared prediction error (SPE) metric for the prediction residuals; combining the first Hotelling metric and the SPE metric of the prediction residuals to generate one or more combined monitoring metrics for the prediction residuals; determining a second Hotelling metric for the prediction latent variables; and combining the second Hotelling metric for the prediction latent variables with the one or more combined monitoring metrics for the prediction residuals. Numbered Examples – Group 2

[0107] Example 2.1. A method for generating a master predictor model from multidimensional time series data, the method comprising: receiving multidimensional time series data corresponding to multiple original variables from multiple sensors; transforming the multidimensional time series data to a lower dimension to define dimensionality-reduced time series data; extracting multiple latent variables from the dimensionality-reduced time series data by a controller and determining the values ​​of the multiple latent variables in a first time period; initializing a loading matrix corresponding to a set of latent variables among the multiple latent variables by the controller; determining one or more master predictor model parameters by the controller through an iterative process, the iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the dimensionality-reduced time series data and the loading matrix by using an estimation process, the estimation process maximizing the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0108] Example 2.2. According to the method described in Example 2.1, transforming the multidimensional time series data to a lower dimension includes performing singular value decomposition (SVD) on the multidimensional time series data.

[0109] Example 2.3. The method according to any one of Examples 2.1 to 2.2 further includes generating a latent vector autoregressive (LaVAR) model based on one or more determined master predictor model parameters.

[0110] Example 2.4. The method according to any one of Examples 2.1 to 2.3, wherein the master predictor model includes a maximum prediction variance (MPV) objective.

[0111] Example 2.5. The method according to any one of Examples 2.1 to 2.4, wherein the estimation process simultaneously maximizes the covariance of the predicted values ​​of the plurality of latent variables or simultaneously maximizes the covariance of the predicted variability in the predicted values ​​of the plurality of latent variables.

[0112] Example 2.6. The method according to any one of Examples 2.1 to 2.5, wherein calculating the new load matrix includes performing eigenvalue decomposition on the matrix of predicted values ​​of the plurality of latent variables to calculate the new load matrix.

[0113] Example 2.7. The method according to any one of Examples 2.1 to 2.6 further includes determining the number of latent variables in the set of latent variables.

[0114] Example 2.8. According to the method described in Example 2.7, wherein the number of latent variables is determined to capture the target predictable changes in the dimensionality-reduced time series data through the latent variables.

[0115] Example 2.9. The method according to any one of Examples 2.7 to 2.8, wherein the determination of the number of latent variables is based on the target prediction variance ratio (PPV).

[0116] Example 2.10. The method according to any one of Examples 2.1 to 2.9 further includes determining the orthogonal complement of the load matrix to define the static load matrix.

[0117] Example 2.11. The method for generating a master predictor model from multidimensional time series data according to any one of Examples 2.1 to 2.10, further comprising: analyzing the multidimensional time series data corresponding to a dynamic system using the master predictor model; generating one or more monitoring indicators of the prediction residuals and / or latent variables of the analyzed multidimensional time series data; and detecting anomalies in the multidimensional time series data based on the one or more monitoring indicators.

[0118] Example 2.12. According to the method of Example 2.11, generating one or more monitoring indicators includes: a) generating one or more first monitoring indicators of the prediction residual; b) generating one or more second monitoring indicators of the latent variable; and c) generating one or more combined monitoring indicators based on the one or more first monitoring indicators of the prediction residual and the one or more second monitoring indicators of the latent variable.

[0119] Example 2.13. The method according to Example 2.12, wherein the one or more first monitoring indicators of the prediction residual include a first Hotelling indicator, a squared prediction error (SPE) indicator, or a combination of the first Hotelling indicator and the SPE indicator defined for the prediction residual.

[0120] Example 2.14. The method according to any one of Examples 2.12 to 2.13, wherein the one or more second monitoring indicators of the latent variable include a second Hotelling indicator defined for the predictive latent variable.

[0121] Example 2.15. The method according to any one of Examples 2.11 to 2.14, wherein the method further comprises generating one or more overall monitoring indicators for both the prediction residuals and the prediction latent variables by: determining a first Hotelling index for the prediction residuals; determining a squared prediction error (SPE) index for the prediction residuals; combining the first Hotelling index and the SPE index of the prediction residuals to generate one or more combined monitoring indicators for the prediction residuals; determining a second Hotelling index for the prediction latent variables; and combining the second Hotelling index of the prediction latent variables with the one or more combined monitoring indicators for the prediction residuals.

[0122] Example 2.16. A method for identifying anomalies detected in multidimensional time series data using a master predictor model generated by any one of Examples 2.1 to 2.15, the method comprising: determining a prediction error matrix based on the difference between predicted values ​​and actual values ​​of the multidimensional time series data; decomposing the prediction error matrix to identify anomaly directions in the prediction error matrix; determining a reconstruction-based contribution (RBC) matrix, the reconstruction-based contribution matrix including the contribution of each variable to the detected anomaly; and analyzing the RBC matrix to identify the detected anomaly.

[0123] Example 2.17. The method according to Example 2.16, wherein the anomaly direction is used to reconstruct anomaly-free data.

[0124] Example 2.18. According to the method described in Example 2.17, reconstructing the anomaly-free data includes: projecting the prediction error matrix onto the anomaly direction, and subtracting the contribution of the detected anomaly from the multidimensional time series data.

[0125] Example 2.19. A system for generating a master predictor model from multidimensional time series data, the system comprising: a plurality of sensors configured to detect multidimensional time series data corresponding to a plurality of original variables; and a processing unit coupled to the plurality of sensors, the processing unit being configured to: transform the multidimensional time series data to a lower dimension to define dimensionality-reduced time series data; extract a plurality of latent variables from the dimensionality-reduced time series data by a controller and determine the values ​​of the plurality of latent variables within a first time period; initialize a loading matrix corresponding to a set of latent variables among the plurality of latent variables by the controller; and by... The controller determines one or more master predictor model parameters by performing an iterative process comprising: (a) predicting the values ​​of the plurality of latent variables based on the dimensionality-reduced time series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the plurality of latent variables and the predicted values ​​of the plurality of latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge.

[0126] Example 2.20. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: receiving multidimensional time-series data corresponding to multiple original variables from multiple sensors; transforming the multidimensional time-series data to a lower dimension to define dimensionality-reduced time-series data; extracting multiple latent variables from the dimensionality-reduced time-series data and determining the values ​​of the multiple latent variables within a first time period; initializing a loading matrix corresponding to a set of latent variables among the multiple latent variables; determining one or more master predictor model parameters by performing an iterative process, the iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the dimensionality-reduced time-series data and the loading matrix by using an estimation process that maximizes the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge. Detailed Implementation

[0127] The embodiments generally relate to methods, systems, and computer-readable media for dynamic process monitoring and / or fault detection, and in some embodiments for generating one or more master predictor models for dynamic process monitoring and / or fault detection. Some embodiments relate to systems, methods, and computer-readable media for generating master predictor models from multidimensional time series data. A multidimensional time series is a time series with multiple variables that change over time. A master predictor model is a model that predicts the output, behavior, or system variables of a dynamic system. Some embodiments relate to generating master predictor models from multidimensional time series data to analyze data in industrial processes, such as electrical, mechanical, and civil engineering systems (particularly industrial processes requiring preventative maintenance). For example, the methods, systems, and computer-readable media disclosed herein can be applied to monitor, analyze, and predict large amounts of data in systems that record multivariate time series data, such as infrastructure and transportation systems, building heating and cooling systems, and industrial manufacturing processes. The embodiments disclosed herein can be used for quality and / or safety monitoring, fault identification, and / or preventative maintenance.

[0128] Some embodiments relate to a method for generating master predictor models from multidimensional time series data for use in dynamic process monitoring and fault detection. This method may be referred to herein as Latent Vector Autoregressive (LaVAR) estimation with a maximum predictive variance (MPV) objective, or LaVAR-MPV. This method may also be referred to herein as a master predictor analysis (PPA) method. In some embodiments, the method is configured to capture the most predictable variance in a dynamic system.

[0129] In some embodiments, the PPA method begins by finding a projection matrix P to ensure that the latent vector v k This represents all the dynamics of the data. The goal is to make the latent vector v k Rather than predicting Maximizing the covariance between them ensures that the most predictable variance is captured. This process begins by satisfying the orthogonality condition P. T The initial projection matrix P of P = I. Using this initial P, the latent vector v is computed. k And the parameter matrix B is estimated by minimizing the least squares objective function. j This involves solving the parameter matrix using the Moore-Penrose pseudoinverse. The term "calculate" as used in this paper, and other forms such as "calculating," "calculated," etc., can refer to calculation, evaluation, determination, and / or estimation, and other forms thereof.

[0130] Then define the MPV objective to leverage Loewner ranking for prediction. The covariance is maximized, thus ensuring that the resulting covariance matrix is ​​a positive definite matrix and, in some embodiments, a maximally positive definite matrix. This is achieved by performing eigenvalue decomposition on the covariance matrix and iteratively updating the projection matrix P until convergence. The algorithm iterates through these steps: updating the projection matrix P and recalculating the latent vector v. k This continues until the covariance matrix converges to a solution that maximizes the prediction variance. Then, the final projection matrix P and parameter matrix B are used. j To build a PPA model.

[0131] Figure 1 This is a flowchart of a method 100 for generating a master predictor model from multidimensional time series data, according to some embodiments. Method 100 involves receiving multidimensional time series data from multiple sensors at 110, wherein the data corresponds to various raw variables. Thus, the raw variables correspond to variables in the time series data; these variables may be observable variables of the system and may correspond to data collected from the sensors. Then, at 115, the multidimensional time series data is transformed into reduced-dimensional time series data. Then, at 120, the data is processed by a controller that extracts multiple latent variables and determines the values ​​of the latent variables within a specified initial time period. Latent variables are variables that are not present in the raw variables (e.g., not directly observed variables), but are inferred from one or more raw variables.

[0132] At 130, the controller initializes loading matrices corresponding to a set of extracted latent variables. Loading matrices are matrices used to represent the relationships between observed and latent variables. Loading matrices may contain coefficients, weights, or other values ​​representing the relationships between observed and latent variables. For example, loading matrices may indicate the degree of association between each observed variable and a latent variable. Larger values ​​in the loading matrix indicate a stronger relationship, thus indicating a greater influence of the latent variable in explaining the observed data, while smaller values ​​indicate a weaker relationship between the observed variables, thus indicating a smaller influence of the latent variable in explaining the observed data. Then, at 140, the controller determines one or more master predictor model parameters through an iterative process. The iterative process involves predicting the values ​​of the latent variables at 150 based on the time series data and the initial loading matrices. The prediction is performed using an estimation process that maximizes the covariance between the actual and predicted values ​​of the latent variables.

[0133] Following the prediction at 150, the controller computes a new loading matrix at 160 by incorporating the predicted values ​​of the latent variables. Then, at 170, the initial loading matrix is ​​updated using the new loading matrix. This iterative process of predicting latent variable values, computing the new loading matrix, and updating the loading matrix is ​​repeated until the master predictor model parameters converge to a stable solution. At 180, the method may include determining whether the master predictor model parameters have converged, and if not, reverting to 150. If convergence is determined, the method may include outputting the master predictor model parameters at 190.

[0134] In some embodiments, method 100 further includes outputting a master predictor model. In some embodiments, method 100 further includes generating a latent vector autoregressive model based on one or more determined master predictor model parameters.

[0135] This method ensures that the master predictor model captures the most predictable variance in multidimensional time series data, making it highly effective for dynamic process monitoring and fault detection in complex systems. The iterative nature of the process allows for continuous refinement and optimization of the model parameters, resulting in more accurate and reliable predictions.

[0136] Multidimensional time series data comprises one or more datasets that track multiple variables over time. It can also be referred to as multivariate time series data. Unlike univariate time series, which monitors a single variable, multidimensional time series captures the interactions and dependencies between multiple variables. Multidimensional time series data can be obtained from various fields, including finance, where it can simultaneously track various economic indicators, or environmental science, where it can simultaneously monitor temperature, humidity, and wind speed. In one example, multidimensional time series data can be used to track multiple variables throughout a manufacturing process. For instance, a factory producing electronic devices might periodically collect data on several variables, such as the temperature and humidity levels of the production environment, machine vibration levels, power consumption of machinery, production speed (units produced per hour), and defect rate (number of defective units per batch). Each of these variables is recorded over time, creating a multidimensional time series dataset. By analyzing this data, the factory can identify patterns and correlations. For example, the factory might find that higher humidity levels are associated with an increased defect rate, or that certain vibration levels in the machines predict maintenance needs.

[0137] While the examples above provide a few variables, multidimensional time series data can include hundreds or thousands of variables recorded over a period of time. Analyzing such multidimensional time series data can reveal complex patterns and relationships that are not apparent when examining each variable individually or a limited number of variables. Multidimensional time series data can be used to generate master predictor models, which are configured to perform process monitoring and / or detect or predict faults in dynamic systems. Because the master predictor model is generated from multidimensional time series data produced by dynamic systems, it is able to accurately analyze and predict data related to dynamic systems.

[0138] In one embodiment, a master predictor model can be used to predict failures within a dynamic system (e.g., within an apparatus, machine) and / or process. The master predictor model, generated from multidimensional time-series data of the dynamic system, is configured to analyze multidimensional time-series data collected from various sensors within the dynamic system. For example, in the above example of an electronics factory, sensors may monitor parameters such as vibration, temperature, and humidity levels. The model can predict impending machine failures based on observed increases in vibration levels, as well as increases in temperature and high humidity over the past week. This predictive capability enables proactive maintenance, thereby reducing downtime and preventing potential damage to machinery. In other words, the master predictor model can predict failures within a dynamic system by analyzing multidimensional time-series data corresponding to the dynamic system.

[0139] The master predictor model can also be used to facilitate adjustments to operating parameters in response to its output. For example, if the model indicates that current operating conditions may lead to failure, the dynamic system can adjust automatically. In the example of an electronics factory, the model can indicate impending failures of machines within the factory, and mechanical parameters can be adjusted to address or mitigate these impending failures, for example, by adjusting the machine's operating speed or cooling settings to reduce the risk of failure. This real-time adjustment in response to the master predictor model's output can provide optimal performance for components within the dynamic system, for example, improving the overall efficiency and productivity of the machines.

[0140] Furthermore, the master predictor model can be used to trigger maintenance activities based on its predictions. When the model predicts that maintenance is needed, it can generate an alert for the maintenance team, indicating the nature of the predicted problem and recommended actions. In some embodiments, the model can accurately and specifically predict where maintenance is required. For example, if the model detects patterns of vibration and temperature rise indicating wear on machine bearings, it can guide preventative maintenance to inspect and replace the bearings before failure occurs. This predictive maintenance approach not only extends the lifespan of dynamic systems but also minimizes unexpected failures and associated costs.

[0141] In some embodiments, the master predictor model can be used to analyze data from multiple components within a dynamic system to monitor processes and guide preventative maintenance of the dynamic system. In some embodiments, the output of the master predictor model can be used to monitor the dynamic system and identify correlations leading to system failures. Therefore, by using a master predictor model to monitor a dynamic system, the likelihood of failures within the system can be reduced. In the example of an electronics factory, the master predictor model can be used to analyze data from multiple machines throughout the factory floor to identify broader trends and correlations within the data. For example, the model can identify machines operating under certain environmental conditions (such as high humidity areas) that are more prone to failure. Based on this process monitoring, the model's output can be used to guide preventative maintenance. For example, the factory can adjust the system to enhance the dehumidification system in specific areas, thereby reducing the likelihood of failures and improving the overall operational reliability of the factory.

[0142] In another example, a master predictor model can be used to enhance maintenance, fault prediction, and operational adjustments within an aero-engine. For instance, the master predictor model can be configured to analyze multidimensional time-series data received from various sensors within the engine, such as engine vibration, exhaust temperature, and oil pressure. If the model detects an abnormal increase in vibration and a corresponding rise in exhaust temperature, it can predict an impending turbine blade failure. This prediction can trigger preventative maintenance actions, such as inspecting and potentially replacing affected components before a failure occurs. Furthermore, the output of the master predictor model can be used to automatically adjust operating parameters in real time. For example, the output of the master predictor model can be fed into avionics systems and used to adjust aero-engine operating parameters (e.g., by reducing engine thrust or adjusting the fuel mixture) to reduce the risk of damage under detected stress conditions. By continuously monitoring and analyzing these parameters, the master predictor model is used to assist in the efficient and safe operation of the engine, thereby extending engine life and enhancing overall flight safety.

[0143] In some embodiments, multidimensional time-series data can be received from one or more sensors that monitor various environmental or operational factors. In some embodiments, multidimensional time-series data can be obtained from one or more data storage devices (such as databases). In some embodiments, multidimensional time-series data can be received from one or more integrated platforms or systems that combine data from various sources. In some embodiments, multidimensional time-series data can be obtained from historical data and / or past records. In some embodiments, multidimensional time-series data can be received from devices such as IoT devices that continuously collect data from networked systems. In some embodiments, multidimensional time-series data can be received in real time from one or more sensors.

[0144] In some embodiments, multiple sensors include multiple devices that detect and measure changes in physical properties or the environment (the natural environment of a digital environment) and convert these measurements into signals that can be read and analyzed. Multiple sensors may include any sensor suitable for acquiring data. For example, one or more sensors may be included as part of a monitoring system that tracks and records data when an event occurs. In some embodiments, multiple sensors may include, but are not limited to, one or more of the following: temperature sensors, humidity sensors, pressure sensors, proximity sensors, light sensors, motion sensors, accelerometers, gyroscopes, magnetic sensors, gas sensors, sound sensors, vibration sensors, flow sensors, force sensors, infrared sensors, ultrasonic sensors, pH sensors, optical sensors, strain gauges, level sensors, image sensors, touch sensors, capacitive sensors, inductive sensors, piezoelectric sensors, chemical sensors, biosensors, radiation sensors, color sensors, thermal sensors, position sensors, displacement sensors, velocity sensors, torque sensors, current sensors, voltage sensors, biometric sensors, electromagnetic sensors, radiation sensors, acoustic sensors, specific substance sensors, RFID sensors, LiDAR sensors, and / or CO2 sensors. It should be understood that the specific sensors used to collect multidimensional time series data within the methods and systems disclosed herein will depend on the specific application domain. For example, the multiple sensors used to obtain multidimensional time series data in an industrial processing environment may differ from the multiple sensors used to obtain multidimensional time series data in a transportation system.

[0145] Existing methods for modeling dynamic multidimensional time series include full-dimensional dynamic series modeling. Full-dimensional dynamic series modeling involves analyzing time series data by considering all available variables or dimensions without reducing the number of said variables or dimensions. This approach aims to capture the full complexity and interactions within the dataset, thereby providing a comprehensive understanding of the system's behavior over time. Techniques such as vector autoregression (VAR) or state-space models can be used to process high-dimensional data, enabling the modeling of complex relationships and dependencies between variables. While this approach can provide more detailed insights, it also requires significant computational resources and can be more challenging to interpret due to the sheer volume of information.

[0146] Measurement vector (y) in a dynamic system k A time series is formed, and if there is a non-zero correlation between different time points, the time series is considered serially correlated. A full-dimensional dynamic series can be represented by its best predictor and unpredictable white noise. Various models (such as state-space or autoregressive integral moving average (ARIMA)) can be used for prediction. Additionally, for vector autoregressive (VAR) models, predictions depend on a finite window of past data.

[0147] set up Let k be the measurement vector of a variable in a dynamic system at time k, and let k form a time series. in, If for some j>0, Then the sequence is considered to be sequence-related. If it exists... Make a T y k If the sequences are uncorrelated, then the time series {y} are correlated. k The time series is defined as a reduced-dimensional dynamic (RDD) sequence. Otherwise, the time series is defined as a full-dimensional dynamic (FDD) sequence.

[0148] Full-dimensional dynamic sequences can be best predicted by them. and unpredictable white noise e k Let be represented as shown in equation (1).

[0149]

[0150] Predictor Relying on past data And e k (It is unpredictable white noise, but can also be called innovation) It does not rely on past data. Predictor Various models can be used, such as state-space models or autoregressive integral moving average (ARIMA) models. For vector autoregressive (VAR) models, prediction... Relying on past data The finite window, which is given by equation (2).

[0151]

[0152] Among them, A j 's is usually a full-rank matrix. For A j An estimate of 's can be found using maximum likelihood or least squares.

[0153] Dimensionality Reduction Dynamics (RDD) sequence modeling is a technique used to simplify the analysis of complex time series data by reducing the number of variables or dimensions involved. By reducing dimensionality, the most significant patterns and trends within the data can be identified. The model can then more efficiently capture the underlying dynamics of the system, making it easier to interpret and predict future behavior. Furthermore, the computational load is significantly reduced due to the minimization of the number of variables or dimensions. Therefore, the memory required for the model to perform the analysis is reduced as fewer data points need to be stored and processed. This efficiency not only speeds up computation but also makes it feasible to analyze large datasets using limited computational resources.

[0154] In some embodiments, multidimensional time series data can be transformed into a lower-dimensional space. In some embodiments, multidimensional time series data can be transformed into a lower-dimensional space by projecting it into a lower dimension to define dimensionality-reduced time series data. Dimensionality-reduced time series data can refer to time series data that has been transformed to reduce the number of dimensions or variables within the time series data. That is, dimensionality-reduced time series data can refer to time series data that has been transformed from its original multidimensional form to a lower-dimensional form. In some embodiments, principal component analysis (PCA) can be used to transform multidimensional time series data into a lower-dimensional space. In some embodiments, s-TNE can be used to transform multidimensional time series data into a lower-dimensional space. In some embodiments, transforming multidimensional time series data into a lower-dimensional space can include performing singular value decomposition on the multidimensional time series data. In some embodiments, the transformation to a lower-dimensional space can include data preprocessing. In some embodiments, data preprocessing can include normalizing the data to ensure that all features are on the same scale, for example, by standardizing each feature to make its mean zero and standard deviation one. Any missing values ​​in the dataset can be handled by imputation or removal, and noise can be reduced by applying appropriate filtering techniques. After preprocessing, the covariance matrix of the data is calculated to understand the variance relationships between the variables. Then, eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and eigenvectors, where the eigenvectors represent the directions of maximum variance (called principal components), and the eigenvalues ​​indicate the magnitude of variance in these directions. The eigenvalues ​​are sorted in descending order, and the top k eigenvectors corresponding to the largest eigenvalues ​​are selected as principal components. These principal components act as dynamic latent variables, capturing the most significant variance in the original data. Finally, the original data is projected onto these selected principal components by multiplying the data matrix by the matrix of the selected eigenvectors, resulting in a lower-dimensional representation that captures the most significant variance in the original data.

[0155] Dynamic latent variables (LVs) can be extracted and / or constructed from multidimensional time series data. LVs are hidden or unobserved variables within a system that change over time and affect observable data. These variables cannot be directly measured but can be inferred through statistical models or algorithms. LVs help understand and predict complex high-dimensional time series data by capturing latent time-dependent patterns. By representing predictable content in the data, LVs allow for a more compact representation of dynamic systems by projecting the original high-dimensional data into a lower-dimensional space.

[0156] In some embodiments, values of a plurality of latent variables extracted from time series data are determined within a first time period. The first time period may refer to a specific defined duration during which a controller extracts latent variables from the time series data and determines values of the latent variables. In some embodiments, the first time period may refer to a subset of the time over which the multi-dimensional time series data spans. In some embodiments, the first time period may refer to a time period starting from a defined point, e.g., a time period starting from time k.

[0157] In some embodiments, transforming multi-dimensional time series data to a lower dimension to define reduced-dimension time series data includes performing singular value decomposition (SVD) on the multi-dimensional time series data. Performing SVD may include, for example, factoring or decomposing a matrix into three matrices. The three matrices may correspond to, for example, a rotation matrix, a rescaling matrix, and another rotation matrix. Performing SVD on multi-dimensional time series data may include: transforming the time series data into a matrix A represented as, and decomposing the matrix into a product of three matrices U, Σ, and V. T U and V are orthogonal matrices (or rotation matrices), and Σ is a diagonal matrix (rescaling matrix) containing the singular values of the time series data matrix. Then, a plurality of latent variables can be extracted from the resulting decomposition.

[0158] In the case of an FDD sequence, the predictor has the same dimension as the measurement result y k However, the RDD time series is different from the FDD sequence in that: the variation within the subspace of y k has time dependence or can be predicted based on past and / or historical data. In contrast, for the RDD sequence, compared with the measurement result y k the predictor has a reduced dimension and can be represented by Equation (3).

[0159]

[0160] In Equation 3, represents a predicted latent vector derived from past data and can also be referred to as a predictor, i.e., Here, represents a sequence-independent vector that does not depend on past data, i.e., for all j > 0, The matrix has full column rank l < p. Thus, for any vector orthogonal to P, the relationship a T y k = a T e k holds, such that the vector cannot be predicted from past data. Dimension It can be called Dynamic Latent Dimension (DLD) or the number of dynamic latent variables.

[0161] In some embodiments, the following method can then be used to obtain The prediction. Given If a matrix has full column rank, then there exists another matrix with full column rank. Make R T P = I. Multiply R by the equation in equation (3). T Equation (4) is obtained.

[0162]

[0163] In equation (4), ε k =R T e k This indicates that innovation does not rely on past measurement data. Therefore, Used as v k The optimal predictor.

[0164] The latent predictor model can be formulated as its past values. Any linear or nonlinear function, as represented by equation (5).

[0165]

[0166] In some embodiments, to parameterize the predictor model, a univariate autoregressive model, a latent vector autoregressive model, or a state-space model can be used. Additionally, in some embodiments, a deterministic control sequence {u} with characteristics for identifying the latent system can be used. k The state-space model of}.

[0167] The method described in this paper differs from existing methods (such as those applied in econometrics and statistics), which utilize raw data y. k To predict DLV (i.e., This existing approach results in a loss of dimensionality reduction parameterization, and is therefore non-parsimony.

[0168] In some embodiments, method 100 considers a latent vector autoregressive (LaVAR) model as depicted in equation (6). However, it should be understood that alternative models, such as state-space models, may be used.

[0169]

[0170] When equations (3) and (6) are combined, a reduced-rank VAR (RRVAR) model is obtained in equation (7), where the VAR parameter matrix {A} j =PBj P T It has a reduced rank.

[0171]

[0172] Matrix P due to its relationship with There is a bilinear relationship but it cannot be determined from the data y k It is uniquely identified in the bilinear decomposition. Therefore, in some embodiments, specific requirements can be imposed on the bilinear decomposition. In some embodiments, v k This can be achieved via tilt projection with unit covariance. In some embodiments, orthogonal P (i.e., Pi) can be identified. T P = I), so that size information is preserved in The middle is maximized. Therefore, it is possible to select... As the orthogonal complement of P, They are orthogonal, that is, they make equation (8) hold.

[0173]

[0174] Using this relationship, equation (3) can then be decomposed using equation (4) to produce equation (9), where ε k =P T e k and It is sequence-independent noise.

[0175]

[0176] Equation (9) indicates that the time-related content in the data is fully contained in v. k In the middle. For i = 1, 2, ..., s, the VAR parameters are... and load P, From data y k The estimate is as follows.

[0177] After extracting multiple latent variables from multidimensional time series data, a loading matrix is ​​initialized, corresponding to a set of latent variables from the multiple latent variables. The loading matrix indicates the relationship between the observed variables and the latent variables. Each element in the loading matrix indicates the weight or contribution of a latent variable to the observed variable. In other words, the loading matrix describes the degree of influence of each latent variable on the observed data. The loading matrix can be represented by P.

[0178] In some embodiments, the method for generating the master predictor model uses LaVAR modeling with the aim of finding make the latent vector This represents all the dynamics of the data. From equations (4) and (9), it can be seen that the LaVAR model will... k Projected onto a lower dimension v k , and then make v k Its past values ​​are the most predictable. As can be seen from equation (7), the parameter matrix is ​​of a multilinear form. Therefore, in some embodiments, the method can utilize an alternating optimization approach.

[0179] After initializing the loading matrix, one or more master predictor model parameters can be determined. Model parameters define the behavior of the model and its ability to make accurate predictions. The model parameters of the master predictor model enable the model to adjust over time to improve prediction accuracy. In some embodiments, these parameters may include coefficients. In some embodiments, coefficients may be adjusted based on new data, allowing the model to dynamically update its predictions. For example, in time series forecasting, parameters may include weights of the most recent observations that change with the addition of new data points. Additionally, parameters may include decay factors to gradually reduce the influence of older data. In some embodiments, parameters may include any parameters suitable for the model. For example, parameters may include, but are not limited to: weights, cluster centroids, decision thresholds, transition probabilities, latent factors, kernel parameters, bias terms, principal components, node weights, eigenvalues ​​and eigenvectors, transition and emission probabilities, basis functions, mean and / or covariance. In this paper, model parameters can be defined by {B} j}express.

[0180] Model parameters can be determined using an iterative process. This process may involve predicting the values ​​of multiple latent variables. These predictions are based on time-series data and an initial loading matrix. The estimation process used in this paper aims to maximize the covariance between the actual values ​​and their predicted values ​​of the latent variables.

[0181] Then, a new loading matrix is ​​calculated. This new matrix is ​​derived from the initial loading matrix and the predicted values ​​of the latent variables. The process begins by obtaining an initial loading matrix, which represents an initial hypothesis or estimate of the relationship between the observed data and the latent variables. Using the predicted values ​​of the latent variables, the controller recalculates the loading matrix to better reflect the current state of the data. After calculating the new loading matrix, the process involves updating the loading matrix with the new loading matrix. That is, the new loading matrix becomes the updated loading matrix, which is then used in subsequent iterations.

[0182] The predicted values ​​are then calculated based on the updated loading matrix, and the process is repeated to calculate a new loading matrix. The following steps—predicting values, calculating the new loading matrix, and updating the loading matrix with the new loading matrix—are repeated until one or more master predictor model parameters converge or reach convergence. In some embodiments, convergence may refer to a point where the master predictor model parameters are stable. For example, this may mean that after several iterations of predicting latent variable values, calculating the loading matrix, and updating the loading matrix, the changes in the parameters become negligible. That is, further iterations do not significantly change the values ​​of these parameters, meaning that the parameters have reached a steady state where further iterations do not significantly change the parameters.

[0183] In some embodiments, convergence is achieved when the model parameters reach a steady state, indicating that the model has reached its optimal state. In some embodiments, this can indicate that the model has adequately captured latent patterns and relationships within the data. In some embodiments, convergence can be determined by setting a predefined threshold for the variation of the model parameters, such that convergence is determined when the difference between successive iterations falls below this threshold. In some embodiments, convergence can be determined by defining a maximum number of iterations, such that if the process reaches the maximum number of iterations, convergence is assumed even if the parameters have not yet fully stabilized. In some embodiments, convergence can be determined by observing when the value of the objective function is optimized, and convergence can further include determining convergence when the variation in the value of the objective function between iterations falls below a certain threshold. In some embodiments, convergence can be determined by monitoring the norm of the gradient (or derivative) of the objective function, such that when the norm becomes sufficiently small, this indicates that the parameters no longer change significantly, and convergence is assumed. In some embodiments, convergence can be determined by checking the stability of the parameters, and convergence is assumed if the parameters do not change significantly within a set number of iterations. In some embodiments, a likelihood ratio test can be used to determine whether the model parameters have converged by comparing the likelihood of successive iterations. In some embodiments, cross-validation techniques can be used to evaluate the performance of the model on different subsets of the data to determine convergence, and convergence is assumed when the performance metric is stable.

[0184] Upon convergence, the iterative process stops when it is determined that the model has reached its optimal configuration. In some embodiments, the optimal configuration may indicate that additional iterations will not provide any substantial changes and / or improvements to the model parameters.

[0185] In some embodiments, it is assumed that P is selected to satisfy P T If P = I, then {v} can be calculated by minimizing the least squares objective. k} to estimate the parameters {B} of the PPA model j}, as shown in equation (10).

[0186]

[0187] Among them, ||·|| F Let Frobenius norm be denoted as , and:

[0188]

[0189]

[0190] V i =[v i+1 v i+2 … v i+N ] T =Y i R i=0,1,…,s (13)

[0191] Among them, Y i =[y i+1 y i+2 … y i+N ] T And Y i It is Y = [y1 y2 … y N … y s+N ] T Submatrices in.

[0192] It can be found in equation (10) The solution is given by equation (14), where, It is a Moore-Penrose pseudo-reverse.

[0193]

[0194] Predicted V s Given by equation (15), then due to the use of Therefore, we obtain equation (16).

[0195]

[0196]

[0197] In some embodiments, the method for generating the master predictor model may include an MPV objective. The MPV objective in a LaVAR model aims to maximize the covariance of the model's predictions. Maximizing the covariance of predictions in the model enhances the model's ability to capture relationships and dependencies between variables. When covariance is maximized, the model more accurately reflects the true variability and interconnections within the data. This enables the model to make more accurate and reliable predictions because it can consider a wider range of possible future scenarios. In other words, the MPV objective ensures that predictions are not focused on a small set of outcomes but can consider various potential states, thereby improving the overall performance and flexibility of the model to accurately capture patterns within multidimensional time series data.

[0198] In some embodiments, the MPV objective is included in the estimation process of method 100. For example, the estimation process may maximize the covariance between the values ​​of multiple latent variables and the predicted values ​​of the multiple latent variables. In some embodiments, the estimation process simultaneously maximizes the covariance of the predicted values ​​of the multiple latent variables. In some embodiments, the estimation process simultaneously maximizes the covariance of the predicted variability in the predicted values ​​of the multiple latent variables.

[0199] In some embodiments, the MPV objective may include: using equation (16) and given an initial P, such that the prediction The covariance is maximized. In some embodiments, this can be maximized using the Loewner order, as given by equation (17).

[0200] stP T P = I

[0201] Notation This means that for all possible P satisfying equation (17), the obtained It is the maximum positive definite.

[0202] In some embodiments, the MPV target makes v k Rather than predicting Maximizing the covariance between them. Alternatively or additionally, in some embodiments, the MPV objective makes the prediction The MPV objective can be implemented similarly to partial least squares (PLS), which maximizes the covariance between two data matrices. In some embodiments, the MPV objective also maximizes the covariance of prediction variability in the DLV.

[0203] In some embodiments, the following method is used to iteratively solve P according to the LaVAR-MPV objective.

[0204] Using the initial Calculate according to equation (14) Perform eigenvalue decomposition on equation (18).

[0205]

[0206] Here, Λ contains eigenvalues ​​that are not in ascending order. By updating equation (19) and iterating the above process until convergence, the optimal solution of (17) can be found.

[0207]

[0208] After convergence, equation (20) contains The variance of each DLV. In some embodiments, after convergence, the method may include determining the orthogonal complement of the load matrix to define the static load matrix. In some embodiments, the following can be selected:

[0209]

[0210] because,

[0211]

[0212] Therefore, it can be seen from (17), (18) and (19):

[0213]

[0214] Typically, P can be linearly represented by W because W forms an orthogonal basis. To make... Orthogonal For a maximum positive definite value, we can choose P such that... That is, the leading eigenvalues There are eigenvectors. Therefore, after convergence, equation (20) holds. We can choose... Make it orthogonal to P.

[0215] Since the rank of the matrix in equation (18) is not greater than 1 Therefore, it exceeds All eigenvalues ​​are zero. Therefore, choose No specific physical meaning of the static load is given. In some embodiments, the static changes are decomposed similarly to PCA, which results in a sequential decrease in the capture variance of the static components. For example, representing Furthermore, it can be seen that due to orthogonal projection, The rank does not exceed Therefore, it is possible to select the right pair Equations (23) through (27) are subjected to SVD. The solution provided by equations (23) through (27) satisfies equation (9).

[0216]

[0217]

[0218]

[0219]

[0220]

[0221] To initialize the algorithm, P is chosen as Y. s The former Principal component loads.

[0222] In some embodiments, the master predictor model can be generated using the following method. First, the method forms a matrix Y from time series data. s This ensures that the matrix has zero mean and unit variance. Next, the method is applied to Y. s Perform singular value decomposition (SVD) to obtain Then, the method uses The former The column is used to initialize P. During the iterative computation phase, the method updates V by multiplying Y by P, forming a new matrix. Then, eigenvalue decomposition is performed to update P. The method then repeats the iterative computation phase until convergence. In some embodiments, the method performs SVD on the converged matrix and selects parameters, setting the values ​​to zero if they fall below a certain threshold.

[0223] In some embodiments, the method for generating the master predictor model is defined by Algorithm 1, which is defined as follows: 1. By form and Y s Scale to zero mean and unit variance. Depend on form And according to Y s The scaling factor is used to scale Y. 2. Regarding Y s Execute SVD as follows: And for the selected initialization 3. Repetition: Calculate V = YP, and form from V Perform eigenvalue decomposition in equation (18) and calculate 4. Until convergence. 5. To Execute SVD and select

[0224] In some embodiments, method 100 further includes determining the number of latent variables in a set of latent variables. In some embodiments, determining the number of latent variables involves capturing a target predictable amount of change in the dimensionality-reduced time series data through latent variables. In some embodiments, the number of latent variables may be selected. The goal is to capture the most predictable variations in the data through DLV, leaving almost no predictable variations in the residuals. In some embodiments, such as process monitoring, the proportion of predicted variance (PPV) is used to select the number of DLVs. That is, in some embodiments, the number of latent variables is determined based on a target cumulative predicted variance percentage. Corresponding prediction The cumulative predictive variance (CPV) of each DLV is:

[0225]

[0226] if To represent the cumulative prediction variance of all p DLVs, the PPV is:

[0227]

[0228] In some embodiments, the generated master predictor model can be constructed using as many DLVs as there are existing dimensions (i.e., using up to p DLVs). Since P is orthogonal, the cumulative prediction variance based on p DLVs according to equation (20) is:

[0229]

[0230] Leading The cumulative predictive variance percentage for each DLV is given below:

[0231]

[0232] In some embodiments, to determine the number of DLVs, one can select... This results in a predefined percentage of the total predicted variance using all p latent variables, where PPV(l) represents at least 95% of the total predicted variance. In some embodiments, the predefined percentage may define the majority of the total predicted variance. In some embodiments, the number of latent variables is determined such that the target PPV represents at least 95% of the total predicted variance.

[0233] The master predictor model is the one selected. A DLV is generated, and in some embodiments, the master predictor model can be used for process monitoring and / or fault detection.

[0234] Since the PPA model aims to maximize the prediction variance, the cumulative prediction variance of the PPA model is no less than the cumulative prediction variance extracted by other latent variable methods with the same number of DLVs.

[0235] DLV Prediction Depends on the past v as given in equation (6) k This further depends on the past y k According to equation (7), the prediction error is:

[0236]

[0237] If the fault affects the data, this is represented as:

[0238]

[0239] Among them, f k It is an actual fault, and It is y k The fault-free portion. The effect of faults on the residuals is given by equation (30), where, It is e k The fault-free part, and q -1 It is a backward shift operator.

[0240]

[0241] As can be seen from equation (30), the impact of faults on prediction errors is filtered out, which can reduce the sensitivity of their detection.

[0242] To illustrate the reduced sensitivity, the following example is provided, where s = 1, B1 = 0.5, and P =

[10] T Therefore, equation (30) gives:

[0243]

[0244] If f 1k If a step fault exists on e, then its effect on e is... 1k The effect of is 0.5 in steady state, while for y 1k The effect is 1.0. Therefore, if e is used... 1k If fault detection is performed, then even if y is removed... 1k The autocorrelation in the data will also reduce the sensitivity to faults.

[0245] For use based on Build a PCA model and based on Existing methods that construct another PCA model to derive monitoring metrics for each of these models lose sensitivity to certain types of failures due to residualization. Another existing method using extended multi-step advance prediction of residuals relies on the successful detection of one-step advance prediction of residuals to switch to extended prediction of residuals.

[0246] In some embodiments, the method for generating a master predictor model may include monitoring residuals and predicting latent variables. In some embodiments, a method is provided for monitoring using a master predictor model generated based on equation (3), the method including monitoring the residuals {e} in equation (3). k} and predictable DLV

[0247] In some embodiments, the generated master predictor model can be used to monitor both the prediction residuals and the predictable dynamic latent variables (DLVs). For the prediction residuals, calculating the innovation sequence and using principal component analysis (PCA) can be used to derive one or more monitoring metrics, such as the Hotelling index and the squared prediction error (SPE) index. The Hotelling index can be used to monitor the scores of the principal components, while the SPE index can be used to monitor the residuals. Additionally, one or more combined monitoring metrics are proposed to monitor both the prediction residuals and the predicted DLVs.

[0248] In some embodiments, method 100 may further include using a master predictor model to analyze time series data corresponding to a dynamic system and detect anomalies in the time series data.

[0249] Figure 2A This is a flowchart of a method 200 for detecting anomalies in a dynamic system according to some embodiments. In method 200, at 210, multidimensional time series data corresponding to the dynamic system is analyzed using a generated master predictor model. The master predictor model may be the model output from method 100. At 220, one or more monitoring indicators of the predicted residuals and / or latent variables of the analyzed time series data are generated. Then, at 230, one or more anomalies in the time series data can be detected based on one or more monitoring indicators.

[0250] Figure 2BThis is a process flowchart of method 250 for generating one or more monitoring indicators according to some embodiments. In method 250, generating one or more monitoring indicators includes generating one or more first monitoring indicators of the prediction residuals at 260. Then, at 270, generating one or more second monitoring indicators of the latent variables. Finally, at 280, generating one or more combined monitoring indicators based on the one or more first monitoring indicators of the prediction residuals generated at 260 and the one or more second monitoring indicators of the latent variables generated at 270. In some embodiments, 260 and 270 in method 250 may be performed sequentially or simultaneously. Although method 250 is in Figure 2B The diagram shows 260 performed before 270; however, it should be understood that 260 may be performed after or concurrently with 270. One or more first monitoring metrics for the prediction residuals may include at least one of a first Hotelling metric and a squared prediction error (SPE) metric defined for the prediction residuals. In some embodiments, one or more first monitoring metrics for the prediction residuals include a combination of a first Hotelling metric and an SPE metric. In some embodiments, one or more second monitoring metrics for the latent variables include a second Hotelling metric defined for the prediction latent variables. As used herein, the terms “first,” “second,” etc., do not indicate any order, quantity, or importance, but are used to distinguish one feature from another.

[0251] In some embodiments, method 100 may further include generating one or more overall monitoring metrics for both the predicted residuals and the predicted latent variables. Figure 3A This is a process flowchart of method 300 for generating one or more population monitoring indicators for predictive residuals and predictive latent variables, according to some embodiments. Method 300 includes determining a squared prediction error (SPE) indicator for the predictive residuals at 310. At 320, a first Hotelling indicator for the predictive residuals and the SPE indicator for the predictive residuals are combined to generate one or more combined monitoring indicators for the predictive residuals. At 330, a second Hotelling indicator for the predictive latent variables is determined. Then, at 340, the second Hotelling indicator for the predictive latent variables is combined with one or more combined monitoring indicators for the predictive residuals generated at 320 to generate one or more population monitoring indicators. At 350, one or more population monitoring indicators are output.

[0252] In some embodiments, one or more monitoring indicators can be generated as follows. Given normal data and the generated master predictor model, the innovation sequence {e} can be calculated. k} to obtain And PCA is performed via EVD as shown in equation (31), where Λ e =diag(λ e,1 ,λ e,2 ,…,λe,p The variances are arranged in descending order and include principal components.

[0253]

[0254] By selecting the principal component or the amount of DLV To capture E, for example s Principal component loadings can be obtained by taking 95% of the total variance.

[0255] The main load P can be used e To calculate the score and residual To execute on e k Monitoring. The Hotelling index can be defined as:

[0256]

[0257] in, It is t e,k The covariance matrix. Approximately follows the l e χ with one degree of freedom 2 The distribution, whose upper limit of control is And its confidence level is (1-α).

[0258] residual The following is monitored using the Squared Prediction Error (SPE) metric:

[0259]

[0260] Assume e k It follows a normal distribution, where λ e,i Let be the variance of the i-th residual component, then Approximately follows the upper limit of the control limit. χ 2 (h e ) distribution, among which,

[0261]

[0262]

[0263] In some embodiments, one or more combined monitoring indicators can be generated. For example, combined indicators may include... and Q e It is a combination of metrics, rather than monitoring them individually. Because and Both follow χ 2 The distributions are independent, therefore the control limit of the combined index provided in equation (33) is... And its confidence level is (1-α).

[0264]

[0265] As can be seen from equation (3), the predicted value of DLV can also be monitored. If used for predicted values If a fault occurs in the past data, it can enhance the detection of such faults.

[0266] We can perform the following based on Theorem 1 Monitoring. Due to the number of DLVs. Selected to include significantly predictable DLVs, therefore predicting The covariance conditions are favorable. Therefore, monitoring of predicted DLV can be implemented using the Hotelling index of equation (34), the control limit of which is... And its confidence level is (1-α).

[0267]

[0268] In some embodiments, one or more overall monitoring metrics can be implemented for both predicted residuals and predicted DLV. Utilizing prediction The fact that it is unrelated to its residuals can be shown in equation (35) by... With φ e Combining these indicators to calculate one or more overall monitoring indicators, the control limits of which are: And its confidence level is (1-α).

[0269]

[0270] The method for generating a master predictor model described in this paper significantly enhances monitoring metrics by capturing the most predictable variance in high-dimensional time series data. An iterative process of continuously refining model parameters allows the master predictor model to adapt to changes in the data. By maximizing the covariance between the latent vector and its predictions, the method provides more accurate and reliable monitoring metrics. These enhanced metrics can improve process monitoring and fault detection because they can more effectively identify anomalies and deviations in complex time series data, such as data provided by industrial and operational dynamic systems.

[0271] In some embodiments, the PPA methods and models disclosed herein can be further implemented with a fault reconstruction process to improve fault diagnosis during fault monitoring. The fault reconstruction process estimates a fault matrix based on fault data and reconstructs data that resembles normal operating data without interference. This fault reconstruction process may include identifying the impact of faults on the system and isolating faults from normal operating data. Therefore, the reconstructed data can represent the behavior of the system under substantially normal operating conditions—that is, the behavior of the system when it is unaffected by the identified faults. Fault reconstruction advantageously provides accurate fault diagnosis by understanding the true performance of the system without distortion caused by faults. The reconstruction process enables the methods and systems to precisely pinpoint the nature and location of faults, thereby facilitating effective corrective actions.

[0272] The fault reconstruction process extends traditional PCA-based reconstruction methods to master predictor analysis by incorporating the SVD of the prediction residuals to estimate the fault direction. In the PPA method, SVD is used to decompose the prediction residuals, which represent the difference between the actual and predicted values. This decomposition helps identify the most prominent directions of the fault. Therefore, it is possible to understand how the fault affects the system and isolate the effects of the fault by analyzing these fault directions.

[0273] In some embodiments, {y k The system remains fault-free up to time k, and fault i affects subsequent data. As shown in equation (36), where, It is the fault-free part of the data and is unknown after a fault occurs.

[0274]

[0275] Then, in order to estimate the fault matrix based on the fault data PCA-based reconstruction can be extended. Equation (37) shows that, given data up to time k, the prediction residual at time k+j Here, y k+j These are the observation data at time k+j. It is the load matrix, and It is a j-step advance prediction of a fault-free DLV. Essentially, equation (37) calculates the difference between the observed data and the predicted data, thus providing a residual indicating the presence of a fault.

[0276]

[0277]

[0278] Equation (38) will predict the residuals Decomposed into two parts: fault-free residuals (in, And the impact of the fault i f k+j Here, Ξ i f k+j This represents the residual that would exist in the absence of faults. Ξ i This is the fault direction matrix, representing the fault direction. The fault direction matrix is ​​obtained by analyzing the prediction residual matrix E. m The fault direction matrix is ​​obtained by performing SVD, and its columns are the left singular vectors corresponding to the maximum singular values, indicating the direction in which the fault is most prominent. k+j It is the fault size at time k+j. Equation (38) demonstrates how the observed residuals are affected by the fault, thus allowing estimation of the fault's impact on the system.

[0279] Then, use equation (39) to predict the residual E m Decompose it. In equation (39), E m It is a matrix representing the prediction residuals during the period affected by the fault, and is composed of... The columns (which are the prediction residuals at time k+j given data up to time k) are constructed. m It can be represented as It is a matrix representing the fault-free portion of the predicted residuals, and is composed of... The columns (which are the prediction residuals that exist in the case of no faults) are constructed. It can be represented as F m It is a matrix representing the magnitude of the fault during the period affected by the fault, and is composed of f k+j It is constructed from columns. F m It can be represented as F m =[f k+j … f k+j ].

[0280]

[0281] Therefore, equation (39) will predict the residual E. m Decomposed into two parts: fault-free residuals And the impact of the fault i F m When the fault size f k+j When it becomes significant, the normal prediction residual Relatively insignificant, therefore, Ξ i and E m Approximate shared column space. This is achieved through... When performing SVD, the estimated fault direction can be set to...

[0282] In some embodiments, the settings are incremented. Make the residual E in equation (39) m It can be reconfigured within its normal control limits. It can be expressed as equation (40).

[0283]

[0284] In some embodiments, reconstruction enables Minimize, thus producing equations (41) and (42).

[0285]

[0286]

[0287] This is a matrix representing the reconstructed fault-free residuals during the period affected by the fault. As shown in equation (40), The result is obtained by subtracting the estimated fault impact from the observed residuals. It is a matrix representing the reconstructed fault size during the period affected by the fault, and is calculated according to equation (41), which projects the residuals onto the fault direction to estimate the fault size.

[0288] In some embodiments, Represented as The j-th column can be used calculate And calculate using equation (33) This means that it can be increased Messenger Return to normal control limits. In some embodiments, it can be determined that... Return to the minimum of normal control limits This is used to determine The reconstruction-based approach.

[0289] Then, the reconstruction-based contribution (RBC) is calculated to quantify the contribution of each variable to the fault. In some embodiments, this involves using the estimated fault direction obtained by SVD on the predicted residuals. The RBC value is derived by projecting the residuals onto these fault directions, where the RBC value indicates the degree of contribution of each variable to the fault.

[0290] In determining validity After that, you can use The columns are used to define the faulty RBC, as in equation (44) (also known as the RBC equation). mThrough E m The diagonal matrix of singular values ​​obtained by SVD. The singular values ​​represent the fault size in each direction. Notation Instructions to take before The square of the singular value corresponding to each fault direction. Used to project the fault contribution back onto the original variable space. The RBC equation calculates the contribution of each variable to the fault by projecting the fault direction onto the original variable space and scaling it according to the fault size in each direction.

[0291]

[0292] Advantageously, these contributions help identify specific variables that cause failures within dynamic systems, thus allowing for more effective targeting of failure causes.

[0293] In some embodiments, the fault reconstructing process may include generating reconstructed data by subtracting an estimated fault impact from observed data. In some embodiments, generating reconstructed data may include estimating the impact of the fault on the system using the fault direction and RBC value. Once the fault impact is estimated, it can then be subtracted from the observed data to generate reconstructed data representing the normal or standard operation of the system. This reconstructed data can substantially eliminate the distortion caused by the fault, thereby clearly showing the normal performance of the dynamic system.

[0294] The methods and systems described in this paper offer several advantages for dynamic process monitoring and fault detection. They capture the most predictable variance in dynamic systems, ensuring the monitoring of the most significant changes. This results in more accurate and reliable predictions. Furthermore, the methods combine the advantages of dynamic latent variable models and principal component analysis, providing a comprehensive monitoring framework capable of handling both predictable and unpredictable changes. Additionally, the methods described are designed to effectively handle noisy and high-dimensional data.

[0295] Figure 3B This is a process flowchart of a PPA-based modeling and monitoring method 360 according to some embodiments. Method 360 may include using the generated master predictor model or PPA model for offline and / or online monitoring.

[0296] During the offline monitoring phase, historical data is used to train the master predictor model to establish a baseline for normal operation. At point 362, the training data is preprocessed. This may include handling missing values ​​using linear interpolation and detecting outliers using the initial PCA model. Outliers exceeding control limits with high confidence levels are disposed of to maintain the continuity of the time series. At point 364, once the training data has been preprocessed at point 362, the hyperparameters of the PPA model are set. In some embodiments, setting the hyperparameters of the PPA model may include selecting the number of DLVs to capture the most predictable variations in the data. For example, the number of DLVs may be selected as a high proportion of the total predictable variance to ensure that the model is neither over-parameterized nor under-parameterized. In some embodiments, setting the hyperparameters of the PPA model may include performing all or part of method 100.

[0297] At point 366, with hyperparameters set, the PPA model is applied to the preprocessed training data to train the PPA model. At point 368, the trained PPA model is then used to generate normal prediction results. In some embodiments, generating normal prediction results may include calculating the prediction residuals of the training data and the master predictor. The prediction residuals represent the difference between actual and predicted values, while the master predictor captures predictable variations in the data. The normal prediction results establish a baseline for the normal operation of the system.

[0298] During the online monitoring phase, real-time test data is collected from the dynamic system. Real-time data can be continuously collected from sensors and other data sources to provide up-to-date information about the current state of the dynamic system.

[0299] At 370, real-time test data is collected from the dynamic system. At 372, similar to the preprocessing of the training set, the real-time test data may also be preprocessed. Preprocessing the real-time data may include handling any missing values ​​and detecting outliers. In some embodiments, linear interpolation may be used to replace missing values, and / or an initial PCA model may be built to detect and handle outliers. At 374, once the real-time test data has been preprocessed at 372, the trained PPA model is applied to compute predictions. In some embodiments, this may include using the PPA model to generate prediction residuals and a master predictor for the incoming data. The results may be used to compare the real-time data with an established normal operating baseline. In some embodiments, the PPA model may be configured to perform all or part of methods 200, 250, and / or 300 as described herein. At 376, monitoring metrics are computed based on the predictions. This involves developing global predictive monitoring metrics that take into account changes in prediction residuals and the master predictor.

[0300] At points 378 and 380, control limits for the monitoring metrics are calculated. In some embodiments, calculating control limits for the monitoring metrics may include developing globally predictive monitoring metrics that take into account variations in prediction residuals and the master predictor. Control limits may be set based on normal prediction results obtained at point 368, thereby providing thresholds for detecting potential faults in the system. If the monitoring metrics exceed these control limits during online monitoring, it indicates the presence of a potential fault requiring further investigation. In some embodiments, monitoring metrics may be calculated continuously and compared to control limits established during the offline monitoring phase.

[0301] At point 382, ​​it is determined whether the monitored index exceeds the control limits. If the monitored index exceeds the control limits, this indicates a potential fault in the system, and method 360 proceeds to the fault diagnosis phase.

[0302] Once a fault is detected, the fault diagnosis phase is initiated. At 384, the prediction error matrix is ​​calculated. In some embodiments, the prediction error matrix can be calculated by subtracting the predicted values ​​from the actual values. At 386, SVD is then performed on the prediction error matrix. Then, at 388, the reconstruction-based contribution, or RBC, is calculated to identify the contribution of each variable to the detected fault. At 390, the fault is diagnosed. In some embodiments, fault diagnosis may include using RBC to identify fault variables. In some embodiments, fault diagnosis may include analyzing the RBC matrix to determine which variables contribute the most to the fault.

[0303] Figure 4 This is a diagram illustrating an example of a dynamic system with process faults according to some embodiments. The process disturbance measurement in the dynamic process example is... The measured control variables are:

[0304]

[0305] Then, y 1k The variance is calculated as follows

[0306] The measured time series y is analyzed using the PPA model described in this paper. k =[y 1k y 2k ] T The normal process gain is assumed to be K = 0.5, and 100 samples are generated. A process gain failure occurs with K = 1.5, and 100 failure samples are generated to test the effectiveness of the PPA algorithm and to compare it with the existing DiCCA and PCA algorithms.

[0307] Figure 5A It is aimed at Figure 4 The example system in the example uses DLV's φe and φ o A comparison of the monitoring charts based on DiCCA and PCA with the monitoring charts based on DiCCA and PCA. Figure 5A The fault detection results for three methods are illustrated. It can be seen that LaVAR-MPV and DiCCA can effectively detect faults, while PCA (which only focuses on variance) fails to detect faults at all. The LaVAR-MPV model shows φ e and φ o The highest fault detection rates were 87% and 98% for DiCCA and φ, respectively; eDiCCA and φ oDiCCA The fault detection rates were 70% and 69% for PCA and Q, respectively; while for PCA, Q... y and φ y The fault detection rates were only 2% and 1%. Since a process gain fault from 0.5 to 1.5 alters the system dynamics but not the variance of the measured data, the PCA completely fails.

[0308] Figure 5B It is aimed at Figure 5A The example system in the example uses DLV's φ e and φ o Another illustration comparing the monitoring charts with those based on DiCCA and PCA. Figure 5A and Figure 5B The same experimental results were presented, in which, Figure 5B The results are displayed in logarithmic form to base 10. Figure 5B In the middle, Q yPCA Equivalent to Figure 5A Q in y ,and Figure 5B Q in yPCA Equivalent to Figure 5A φ in y .

[0309] In another example, the LaVAR-MPV model algorithm is tested against two other algorithms tested above (i.e., DiCCA and PCA) using a simulated Tennessee Eastman Process (TEP) dataset. PCA is used directly to process the raw data. Six modes of the TEP dataset are provided, representing 28 fault types. In this example, IDV2, IDV11, and IDV13 are selected to demonstrate the effectiveness of the three algorithms in detecting step faults, random faults, and slowly drifting faults, respectively.

[0310] Twenty-three variables were considered, including XMEAS(1) through (22) and XMEAS(40), which involve loops of 19 distributed control strategies. Figure 6This is a table showing the 23 measured variables studied in 19 distributed control loops. The number of DLVs needs to be determined before constructing the LaVAR-MPV model for process monitoring. In this example, computation is performed on a DLV-based model. To choose Using equation (29), select Make To account for at least 95% of the total predictive variance using all 23 possible latent variables. For comparison, for the DiCCA model, select... Make To extract the most representative dynamics from multidimensional time series data.

[0311] Figure 7 It shows LaVAR-MPV, LaVAR-CCA, and DiCCA. The curve is shown. Since CPV is calculated from the training data, more DLV tends to capture a higher CPV. However, as... Figure 7 As shown, when the number of DLVs approaches its maximum, DiCCA captures a lower CPV than the other two methods, which is attributed to the univariate DLV model used in DiCCA. It can also be seen that the CPV of the LaVAR-MPV model is generally greater than that of the LaVAR-CCA model. This is because the LaVAR-MPV model aims to achieve a higher maximum prediction variance.

[0312] Figure 8A The monitoring index φ is shown for the LaVAR-MPV model, as well as two other methods, DiCCA and PCA. e φ o φ eDiCCA φ oDiCCA Q y and φ y The fault detection rate (FDR) and false alarm rate (FAR). φ eDiCCA and φ oDiCCA It is the monitoring indicator φ of the DiCCA model. e and φ o Q y It is the squared prediction error (SPE) of the PCA residuals, and φ y It is the overall metric for the PCA method. The Type I error of the three models built from the normal dataset can be set to 5%, and the α value is 0.05. Figure 8A Indications, regarding both FDR and FAR of these three models, φ o (For PCA as φ) y It showed better performance than φ when monitoring three types of faults. e(For PCA as Q) y Consider monitoring φ of IDV2 and IDV11. e (For PCA as Q) y ), despite Q y The FAR of IDV11 monitoring was significantly higher than the other two methods, but its FDR was higher than the LaVAR-MPV and DiCCA models. φ e Its performance is better than φ eDiCCA For IDV13, LaVAR-MPV has the highest FDR among the three methods, and Q... y The performance shown is weaker than the other two methods, with the smallest FAR. When using φ... o (For PCA as φ) y When monitoring all three fault types, the LaVAR-MPV model remains the best in terms of FDR. DiCCA shows an overall superior FDR compared to PCA. Therefore, this indicates that the LaVAR-MPV model has the best FDR performance. o (With the highest FDR and lowest FAR) is a good choice for monitoring these three fault types because φ o It is a global indicator targeting the potential dynamics in complex, multidimensional time series data.

[0313] Figure 8B The monitoring index φ is shown for the PPA model (or LaVAR-MPV model) and two other methods, DiCCA and PCA. e φ o φ eDiCCA φ oDiCCA Q yPCA (or Q) y ) and φ yPCA (or φ) y Another graph showing the Fault Detection Rate (FDR) and False Alarm Rate (FAR). The graph illustrates: from Figure 8B The conclusions drawn from and from Figure 8A The conclusions drawn are similar.

[0314] Figure 9A It is for IDV2 using 8 DLVs φ e and φ o A comparison of the monitoring charts based on DiCCA and PCA with the monitoring charts based on DiCCA and PCA. Figure 9B It is aimed at Figure 9A IDV2 uses 8 DLVs with φ e and φ o Another illustration comparing the monitoring charts with those based on DiCCA and PCA. Figure 9B The results after taking the logarithm to base 10 are presented, and plotted. Figure 9AThe results of the first 800 or so experiments. Figure 10 It is for IDV11 using 8 DLVs φ e and φ o A comparison of the monitoring charts based on DiCCA and PCA with the monitoring charts based on DiCCA and PCA. Figure 11 It is for IDV13 using 8 DLVs φ e and φ o A comparison of monitoring charts based on DiCCA and PCA is shown. Clearly, each of these three methods can distinguish normal data from fault data with different characteristics, because the impact of faults in simulated data is much greater than in normal conditions. However, it should be noted that... Figure 8A and Figure 8B The results show slight differences in detection effectiveness.

[0315] The Dow Challenge dataset, provided by Dow Chemical, was used as a practical test to evaluate the effectiveness of the PPA method described in this paper. This dataset (also referred to as the test dataset) encompasses data from a purification process involving three distillation columns that tend to accumulate impurities due to accelerated catalyst aging. The primary objective was to predict impurity levels in the process. Two distinct datasets were provided for this test. The first dataset spans from December 2015 to January 2017, and the second dataset covers the period from February 2017 to October 2017. These datasets demonstrate variations in operating patterns and anomalies, making them suitable for testing the PPA method.

[0316] The primary columns in the dataset were selected for monitoring and diagnosing faults, utilizing 15 key process variables. These variables included reflux flow rate, wake flow rate, feed flow rate from column 2, manufacturing flow rate, reference level, bed 1 DP, bed 2 DP, bed 3 DP, bed 4 DP, bottom pressure, top pressure, bed 4 temperature, bed 3 temperature, bed 2 temperature, and bed 1 temperature. Four additional variables with negligible variation were excluded from the analysis. The data period from January to May 2016 contained fewer anomalies and was used to build the model for monitoring purposes. The subsequent 15 days were used for monitoring and diagnosing anomalies and disturbances.

[0317] Although the training data in the test dataset is considered to come from normal operation, it contains missing values ​​and outliers. These missing and outlier values ​​may be preprocessed before building the normal model. Therefore, as part of testing the proposed PPA algorithm on the test dataset, a dynamic model is used to reconstruct or regenerate missing or outlier samples to maintain the integrity of the time series. The following steps are used to preprocess and organize the refined training set for subsequent modeling: • Linear interpolation: Uses linear interpolation to replace missing values ​​where there are valid values ​​both before and after.

[0318] Initial PCA Model: An initial PCA model is constructed to generate the Q-indicator, which is used to detect outliers exceeding the control limits with very high confidence (99.9%). Detected sporadic high outliers are further processed using the same procedure as in the previous steps.

[0319] PPA Model: A PPA model is built based on training data preprocessed with PCA to use φ e Monitoring indicators detect additional outliers. This step helps distinguish between normal dynamic changes and anomalous operations. For example, it identifies periods of high Q values ​​detected by PCA that are caused by normal dynamics, while further analyzing periods caused by anomalous operations.

[0320] The effectiveness of the PPA model in preprocessing is demonstrated by comparing the predicted and actual values ​​of key variables (such as "bed 4DP"). Significant deviations between the actual and predicted values ​​indicate anomalies, which are then corrected using the predicted values ​​to construct a refined PPA model for monitoring and diagnosis. The resulting refined PPA model has a φ... e The monitored indicators were determined to be normal (except for a few isolated outliers), resulting in an acceptable Type I error rate.

[0321] Figure 12 The graphs show the PTV (top) and PPV (bottom) of the PPA, LaVAR, and DiCCA models on the test dataset. Figure 13 This shows the monitoring index φ of the PPA model for interference 1 and interference 2. e The curve graph. Figure 14 This is a graph showing the training and test data up to June 15th, separated by a vertical dashed line. The periods of interference detected by the two PPAs are highlighted in different shades. Figure 15 This is a statistical chart showing the contribution of each variable to interference 1 and interference 2, respectively. Figure 16 The monitoring indicators for interference 1 and interference 2 are shown. and φ e The curve graph. Figure 17 This is a set of graphs showing the original values ​​and PPA generated values ​​for anomalous periods in the training and test datasets.

[0322] Select the number of PPADLV Of these, PPV accounts for at least 95% of the total predictable variance using 15 latent variables. Figure 17 A PPA model for preprocessing training data is given. and It can be seen that, This is sufficient to account for 95% of the PPV in the final PPA model. For comparison, Figure 17 The image also shows LaVAR-CCA and DiCCA. and Clearly, among these three models, PPA achieves the highest PTV with the same number of DLVs. This result is because the covariance objective is maximized in the PPA model, while canonical correlation is maximized in the other two models.

[0323] After obtaining the final master predictor analysis (PPA) model and the corresponding monitoring and control limits, the model is applied to perform fault detection and diagnosis on the test dataset from June 1 to June 15. Figure 13 The monitoring charts for the test set shown indicate the monitoring metric φ e The values ​​were very high at two different time points, exceeding the control limits. To verify the significant impact of these disturbances on product impurities, Figure 14 The training and test sets contain impurity samples. These two time periods are labeled as Interference 1 and Interference 2 for further diagnosis of their contributing factors.

[0324] For the detected interference period, perform fault diagnosis to identify the fault source and implement fault reconstruction using equations (42) and (43). Figure 15 The RBCs for each variable with respect to interference 1 and interference 2 are presented. The two interferences exhibit very different characteristics. While "manufacturing flow rate" and "reference level" are the dominant variables contributing to interference 2, almost all variables appear to contribute consistently to interference 1, with "manufacturing flow rate" and "feed flow rate from FC" being the main contributing factors.

[0325] To determine whether the fault monitoring index φ can be fully utilized e The appropriate number of fault directions to reconstruct back to the normal region. Figure 16 The logarithmic scale shows the reconstruction for these two types of disturbances. and the original φ e The PPA-based reconstruction method uses separate methods for interference 1 and interference 2. and Effectively will basically all The values ​​have all returned to within the normal control limits. The number of DLVs is used for calculation. Figure 15 The RBC shown.

[0326] Using the reconstruction in equation (41), normal operating data without interference can be generated. Using Y... m =[y k+1 … y k+m ] indicates abnormal data The reconstructed data for the specified time period is given by equation (45), where,

[0327]

[0328] Reconstructed data for the observed disturbance periods in the training set and two disturbances from the test dataset are generated and compared with their actual values, such as... Figure 17 As shown, the original values ​​of these three disturbances show significant differences from their normal state, but the reconstructed values ​​appear to be closer to the normal data, demonstrating the effectiveness of the proposed reconstruction method for data cleansing.

[0329] The method described in this paper for generating master predictor models, or PPA method, offers advancements in high-dimensional data modeling by addressing the limitations of traditional univariate latent autoregressive models. By employing a vector autoregressive model on all latent variables and maximizing the covariance between the latent vectors and their predictions, the PPA method simultaneously extracts dynamically decreasing latent variables with decreasing predictability while ensuring the capture of the most predictable variance. The iterative estimation process used in the PPA method continuously refines and optimizes the model parameters, resulting in more accurate and reliable predictions from the generated model over time. This optimization and improved accuracy enhance computational power, making the method faster and more efficient when processing high-dimensional time series data. Therefore, the PPA method is highly effective and efficient for dynamic process monitoring, fault detection, and various industrial applications. Furthermore, the PPA method requires only a limited set of past data to generate the model. This means that the PPA method can make predictions without accessing all past data of the system, thereby reducing the memory required for analysis.

[0330] Figure 18 This is a block diagram of a system 1800 configured to generate one or more master predictor models according to some embodiments. In some embodiments, system 1800 may also be configured to perform process monitoring and / or fault detection using the generated master predictor models. In some embodiments, system 1800 may be configured to perform any of the methods described herein, including methods 100, 200, 250, and 300. System 1800 includes one or more processors 1802 and memory 1804. The processor(s) 1802 may include integrated electronic circuitry performing computations and may include, for example, a microprocessor. The processor(s) 1802 may include one or more microprocessors, graphics processing units (GPUs), central processing units (CPUs), application-specific instruction set processors (ASIPs), application-specific integrated circuits (ASICs), or other processors capable of reading and executing instruction code.

[0331] In some embodiments, processor 1802 may take the form of a controller, wherein the controller serves as the central processing unit (CPU) of the system. The term "controller" means any device, system, or part thereof that controls at least one operation. A controller may be implemented in hardware, firmware, software, or some combination thereof. The functionality associated with any particular controller may be centralized or distributed, whether performed locally or remotely. A controller may execute instructions from a software program, process data, and manage information flow within the system. A controller may include an arithmetic logic unit (ALU) for performing calculations, a control unit for bootstrapping operations, and a memory unit for storing data and instructions. A controller may be configured to perform complex calculations, manage peripheral devices, and ensure efficient task execution. In some embodiments, a controller may be configured to interact with one or more modules in memory 1804 to retrieve instructions, decode instructions, interact with instructions and / or execute instructions, and / or store results and outputs.

[0332] As used herein, the phrase “configured to” may be used interchangeably with the phrases “suitable for,” “capable of,” “designed to,” “suitable for,” “manufactured as,” or “capable of.” The phrase “configured to” does not inherently imply “specifically designed in hardware.” Rather, the phrase “configured to” may imply that a device can perform operations in conjunction with another device or component. For example, the phrase “processor configured to perform A, B, and C” may mean a general-purpose processor (such as a CPU or application processor) that can perform operations by executing one or more software programs stored in a memory device, or a special-purpose processor (such as an embedded processor) for performing operations.

[0333] Memory 1804 may include one or more volatile or non-volatile memory types. For example, memory 1804 may include one or more of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Memory 1804 includes program code (e.g., configured to store executable code modules or engines) accessible by processor(s). When executed by processor(s) 1802, the program code provides various computational capabilities and functions to system 1800, enabling the system to perform certain functions described herein. In some embodiments, memory 1804 stores instructions (such as program code) that, when executed by processor(s) 1802, cause system 1800 to perform methods for generating one or more master predictor models from multidimensional time series data and / or to operate according to any of the methods described herein (including any of methods 100, 200, 250, and 300).

[0334] In some embodiments, system 1800 may be implemented as a distributed system comprising multiple server systems configured to communicate over a network to provide the functionality of system 1800. For example, one or more of the program code (e.g., modules or engines) may be deployed on one or more different or remote servers that may collaborate to provide the described functionality of system 1800. In some embodiments, system 1800 may communicate with network 1806 and may be configured to facilitate communication with additional components, including sensors (multiple) 1808 and one or more data storage devices 1812. In some embodiments, the system may include a network interface (not shown) that may include a combination of network interface hardware and network interface software suitable for establishing, maintaining, and facilitating communication over relevant communication channels.

[0335] System 1800 can communicate with multiple sensors 1808. Sensors 1808 can be configured to receive data from dynamic system 1810. The received data may be multidimensional time-series data. In some embodiments, sensors 1808 are configured to transmit the received data to system 1800 via network 1806. In some embodiments, the system may be configured to receive data directly from the sensors via wired or wireless connections, and in some embodiments, sensors 1808 may be part of system 1800. One or more data storage devices 1812 may be part of system 1800 or local to the system, or remote from system 1800 and accessible to the system, for example, via network 1806. One or more data storage devices 1812 may be relational or non-relational databases. In some embodiments, data storage devices 1812 may be databases configured to store multidimensional time-series data. In some embodiments, data received by sensors 1808 may be stored in data storage devices 1812 and retrieved by system 1800.

[0336] System 1800 may include a master predictor model generation module 1814 configured to generate one or more master predictor models from multidimensional time series data. Master predictor model generation module 1814 may be executed by processor 1802. Master predictor model generation module 1814 may be configured to perform method 100 as disclosed herein. Generation module 1814 may be configured to receive multidimensional time series data from multiple sensors 1808, transform the multidimensional time series data into reduced-dimensional time series data, and extract multiple latent variables from the time series data. Processor 1802 executing generation module 1814 may determine the values ​​of the multiple latent variables within a first time period, initialize a loading matrix corresponding to a subset of the latent variables of the multiple extracted latent variables, and determine one or more master predictor model parameters by performing an iterative process. In some embodiments, the processor 1802 executing the generation module 1814 may predict the values ​​of multiple latent variables based on time series data and a loading matrix, calculate a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables, update the loading matrix based on the calculated new loading matrix, and iteratively repeat these steps until one or more master predictor model parameters converge. The generation module 1814 may be configured to output the master predictor model to, for example, a data storage device 1812.

[0337] In some embodiments, system 1800 may include a monitoring module 1816 configured to generate one or more monitoring metrics using the generated dynamic predictor module. In some embodiments, monitoring module 1816 may be part of or included within generation module 1814. In some embodiments, monitoring module 1816 may be configured to perform process monitoring and / or fault detection of system 1810. Monitoring module 1816 may be configured to perform any of the methods described herein, including any of methods 200, 250, and 300, via processor 1802. Monitoring module 1816 may be configured to analyze time series data received from multiple sensors 1808 corresponding to dynamic system 1810 via processor 1802 using the generated master predictor model output from generation module 1814. In some embodiments, monitoring module 1816 is configured to generate one or more monitoring metrics of the predicted residuals and / or latent variables of the analyzed time series data and to detect anomalies in the time series data based on the monitoring metrics. The monitoring module 1816 can be further configured to generate one or more monitoring indicators by: generating one or more first monitoring indicators of the predicted residuals; generating one or more second monitoring indicators of the latent variables; and generating one or more combined monitoring indicators based on one or more first monitoring indicators and one or more second monitoring indicators. The system can perform the operations described in the reference above. Figures 1 to 17 Any method described.

[0338] In some embodiments, a machine-readable medium is provided storing instructions that, when executed by one or more processors of a system, cause the system to perform the following operations: receive multidimensional time-series data corresponding to multiple raw variables from multiple sensors; extract multiple latent variables from the time-series data by a controller and determine the values ​​of the multiple latent variables within a first time period; initialize a loading matrix corresponding to a set of latent variables from the multiple latent variables by the controller; and determine one or more master predictor model parameters by the controller through an iterative process comprising: (a) predicting the values ​​of the multiple latent variables based on the time-series data and the loading matrix using an estimation process that maximizes the covariance between the values ​​of the multiple latent variables and the predicted values ​​of the multiple latent variables; (b) calculating a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; (c) updating the loading matrix based on the calculated new loading matrix; and (d) iteratively repeating (a) to (c) until the one or more master predictor model parameters converge. The machine-readable medium storing the instructions may be non-transitory. The instructions may include instructions for causing one or more processors to perform the above-mentioned references. Figures 1 to 17 Instructions for any method described.

[0339] In some embodiments, a machine-readable medium is provided that stores instructions that, when executed by one or more processors of a system, cause the system to perform any of the methods disclosed herein, including any of methods 100, 200, 250, and 300.

[0340] Unless otherwise specified by logic, any feature of any embodiment or appended claim may be combined with features of any other embodiment and / or appended claim.

[0341] Those skilled in the art will understand that various changes and / or modifications can be made to the above embodiments without departing from the broad overall scope of this disclosure. Therefore, the embodiments of this disclosure are to be considered illustrative rather than restrictive in all respects.

Claims

1. A method for generating a master predictor model from multidimensional time series data, the method comprising: Receive multidimensional time series data corresponding to multiple raw variables from multiple sensors; The multidimensional time series data is transformed to a lower dimension to define dimensionality-reduced time series data; The controller extracts multiple latent variables from the dimensionality-reduced time series data and determines the values ​​of the multiple latent variables within a first time period. The controller initializes a load matrix corresponding to a set of latent variables from the plurality of latent variables; The controller determines one or more master predictor model parameters by executing an iterative process, the iterative process including: (a) Based on the reduced-dimensional time series data and the loading matrix, the values ​​of the plurality of latent variables are predicted by using an estimation process that maximizes the covariance between the values ​​of the plurality of latent variables and the predicted values ​​of the plurality of latent variables; (b) Calculate a new load matrix based on the load matrix and the predicted values ​​of the latent variables; (c) Update the load matrix based on the newly calculated load matrix; and (d) Iteratively repeat (a) to (c) until the parameters of the one or more master predictor models converge.

2. The method according to claim 1, wherein, Transforming the multidimensional time series data to a lower dimension includes performing singular value decomposition (SVD) on the multidimensional time series data.

3. The method of claim 1, further comprising generating a latent vector autoregressive (LaVAR) model based on one or more determined master predictor model parameters.

4. The method according to claim 1, wherein, The master predictor model includes a maximum prediction variance (MPV) objective.

5. The method according to claim 1, wherein, The estimation process simultaneously maximizes the covariance of the predicted values ​​of the multiple latent variables or simultaneously maximizes the covariance of the predicted variability among the predicted values ​​of the multiple latent variables.

6. The method according to claim 1, wherein, Calculating the new loading matrix involves performing eigenvalue decomposition on the matrix of predicted values ​​of the plurality of latent variables to compute the new loading matrix.

7. The method of claim 1, further comprising determining the number of latent variables in the set of latent variables.

8. The method according to claim 7, wherein, The number of latent variables is determined to capture the target predictable changes in the dimensionality-reduced time series data.

9. The method according to claim 7, wherein, The number of latent variables was determined based on the target predictive variance proportion (PPV).

10. The method of claim 1, further comprising determining the orthogonal complement of the load matrix to define a static load matrix.

11. The method for generating a master predictor model from multidimensional time series data according to claim 1, further comprising: The master predictor model is used to analyze the multidimensional time series data corresponding to the dynamic system; Generate one or more monitoring indicators for the predicted residuals and / or latent variables of the analyzed multidimensional time series data; and Anomalies in the multidimensional time series data are detected based on one or more monitoring indicators.

12. The method according to claim 11, wherein, Generating one or more monitoring indicators includes: a) Generate one or more first monitoring indicators for the predicted residuals; b) Generate one or more second monitoring indicators for the latent variables; and c) Generate one or more combined monitoring indicators based on the one or more first monitoring indicators of the predicted residuals and the one or more second monitoring indicators of the latent variables.

13. The method according to claim 12, wherein, The one or more first monitoring metrics of the prediction residual include a first Hotelling metric, a squared prediction error (SPE) metric, or a combination of the first Hotelling metric and the SPE metric defined for the prediction residual.

14. The method according to claim 12, wherein, The one or more second monitoring indicators of the latent variable include a second Hotling indicator defined for the predictive latent variable.

15. The method according to claim 11, wherein, The method further includes generating one or more overall monitoring indicators for both the predicted residuals and the predicted latent variables in the following manner: Determine the first Hotelling index for the predicted residual; Determine the squared prediction error (SPE) index of the predicted residuals; The first Hotling index of the predicted residual and the SPE index of the predicted residual are combined to generate one or more combined monitoring indicators of the predicted residual. Determine the second Hotelling index for the predicted latent variables; and The second Hotling index for predicting the latent variable is combined with one or more combined monitoring indices for predicting the residual.

16. A method for identifying anomalies detected in multidimensional time series data using a master predictor model generated by the method of claim 1, the method comprising: The prediction error matrix is ​​determined based on the difference between the predicted values ​​and the actual values ​​of the multidimensional time series data. The prediction error matrix is ​​decomposed to identify abnormal directions within the prediction error matrix; Determine a reconstruction-based contribution (RBC) matrix, which includes the contribution of each variable to the detected anomaly; as well as The RBC matrix is ​​analyzed to identify the detected anomalies.

17. The method according to claim 16, wherein, The abnormal direction is used to reconstruct data without abnormalities.

18. The method according to claim 17, wherein, Reconstructing the anomaly-free data includes: projecting the prediction error matrix onto the anomaly direction, and subtracting the contribution of the detected anomaly from the multidimensional time series data.

19. A system for generating a master predictor model from multidimensional time series data, the system comprising: Multiple sensors, configured to detect multidimensional time-series data corresponding to multiple raw variables; as well as A processing unit coupled to the plurality of sensors, the processing unit being configured to: The multidimensional time series data is transformed to a lower dimension to define dimensionality-reduced time series data; The controller extracts multiple latent variables from the dimensionality-reduced time series data and determines the values ​​of the multiple latent variables within a first time period. The controller initializes a load matrix corresponding to a set of latent variables from the plurality of latent variables; The controller determines one or more master predictor model parameters by executing an iterative process, the iterative process including: (a) Based on the reduced-dimensional time series data and the loading matrix, the values ​​of the plurality of latent variables are predicted by using an estimation process that maximizes the covariance between the values ​​of the plurality of latent variables and the predicted values ​​of the plurality of latent variables; (b) Calculate a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) Update the load matrix based on the newly calculated load matrix; and (d) Iteratively repeat (a) to (c) until the parameters of the one or more master predictor models converge.

20. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: Receive multidimensional time series data corresponding to multiple raw variables from multiple sensors; The multidimensional time series data is transformed to a lower dimension to define dimensionality-reduced time series data; Extract multiple latent variables from the dimensionality-reduced time series data and determine the values ​​of the multiple latent variables within a first time period; Initialize the loading matrix corresponding to a set of latent variables from the plurality of latent variables; One or more master predictor model parameters are determined by performing an iterative process, which includes: (a) Based on the reduced-dimensional time series data and the loading matrix, the values ​​of the plurality of latent variables are predicted by using an estimation process that maximizes the covariance between the values ​​of the plurality of latent variables and the predicted values ​​of the plurality of latent variables; (b) Calculate a new loading matrix based on the loading matrix and the predicted values ​​of the latent variables; and (c) Update the load matrix based on the newly calculated load matrix; and (d) Iteratively repeat (a) to (c) until the parameters of the one or more master predictor models converge.

Images