Adaptive function neural layer and network

Abstract

The present disclosure relates to a method of generating or updating signal processing logic for processing signals, in particular time-series signals, comprising measurements associated with a physical asset, the method comprising: training a machine learning ML model, in particular a function neural network FNN, for generating or updating a signal processing logic for processing signals, in particular time-series signals, comprising measurements associated with a physical asset; wherein the ML model comprises performing the following operations: receiving a processing signal; determining a plurality of first level vectors based on the processed signals and the plurality of basis functions, in particular by projecting each of the processed signals to a respective one of the plurality of basis functions, where each of the plurality of first level vectors is correlated with each projected one of the processed signals by the respective one of the basis functions; and determining a plurality of second level vectors based on the plurality of first level vectors, in particular by projecting each of the plurality of first level vectors to a respective one of the plurality of transformation matrices, wherein each vector of the plurality of second level vectors is associated with each projected vector of the plurality of first level vectors by a respective transformation matrix. The disclosure further relates to a corresponding determination system and an industrial or power system.

Description

Technical Field

[0001] The present disclosure relates to a method of generating or updating signal processing logic for processing signals, particularly time series signals, including measurements associated with physical assets, a determination system, and an industrial or power system. Background Art

[0002] Functional Neural Networks (FNNs) can be used for monitoring, diagnosis and analysis, and can be used as part of the logic for automated decision making for industrial assets. This includes assets used in the power industry, such as generation assets and their components, transmission assets and their components, and distribution assets and their components. For industrial assets, there may be different quantities of interest that need to be monitored and / or estimated, such as operating performance, operating status, or information about external conditions or neighboring systems. The information thus obtained can be used to provide information to human operators, managers, or stakeholders to support their operations or other decisions, or to partially or fully automate the operation of the asset. Industrial assets typically generate measurement data in the form of numerical and / or categorical time series when operating. Such time series can be recorded continuously at regular time intervals (periodic sampling), or at irregular time intervals. Measurements can also be triggered by certain events, for example, a time series can be recorded when certain anomalies in the operating conditions are detected.

[0003] Time series measured and sampled at regular or irregular intervals constitute functional data: this data can be viewed as a function of time. Functional data can be analyzed using functions and can be viewed as consisting of an infinite number of samples, which themselves are also analyzed. Sampling is only a necessary technical property of the data, as it is technically impossible to achieve infinite sampling.

[0004] While time series measurements of an asset at any time during its operation or life may be obtained over a limited time window covering recent history or even over its entire operating history, some quantities of interest may not be directly measurable and need to be derived from the data. In some cases, there may be known physical laws that can be used to calculate the quantity of interest from the measured data, such as for determining the efficiency of an engine. In other cases, typically primarily for the health of the asset, no such physical laws are known to relate the measured data to the quantity of interest. The development of machine learning methods for these cases is an active area of ​​research. In particular, artificial neural networks (ANNs) are a promising candidate technology in this regard.

[0005] Functional Data Analysis is a method for processing functional input data to statistical models. If functional input data is used as input to an ANN in this way, this is called a functional neural network (FNN). Functional Data Analysis processes data in a way that is as independent of the sampling time as possible. In particular, FNN can be applied to time series with irregular sampling and missing values.

[0006] Traditional FNN Figure 1a ). Only the first layer is a function layer, accepting function input. It is labeled This refers to the fact that by feeding the function of all input channels i into f i (t) Projection To the basis function in all basis functions j to compute the output of the layer. (A different basis function may be chosen for each channel i, in which case the projection reads ). Function inputs are usually sampled at discrete times. In this case, the projection can be approximated by

[0007]

[0008] This sum can be replaced by a weighted sum to take into account the irregular sampling intervals. For simplicity of notation, the weight term is not shown here. Figure 1b ) shows a small example. Here, two function inputs are projected onto two basis functions, resulting in a four-dimensional output of the function layer. After the first function layer of the FNN, more traditional multi-layer perceptron (MLP) layers follow. These layers can be of any known type or new types: fully connected, convolutional, recurrent, etc. Figure 1b ) shows one hidden fully connected layer with rectified linear unit (ReLU) activation and one fully connected output layer with linear activation.

[0009] One design decision when building an FNN (in the context of machine learning, such design decisions are often referred to as hyperparameter selection) is the choice of basis functions used in the first functional layer. In this way, the FNN architecture can be implemented with an adaptive base layer. Figure 2 A traditional FNN with an adaptive basis layer (AdaFNN) is illustrated. For simplicity, only the input layer is shown in detail, while the remaining layers (e.g., fully connected MLP, convolutional layers, recurrent layers, etc.) are omitted. The input is visualized in two dimensions: channels (3 channels in this example) and time steps (7 time steps in this example). The time steps do not need to be regularly spaced, and the number of time steps per channel does not need to be the same. Basis Functions can, but need not, depend on channel i. These basis functions can be fixed or adaptive.

[0010] Implementing an adaptive base layer means that the choice of basis functions changes. This means that it is excluded from the hyperparameter selection, which imposes additional work on the modeler, leading to higher modeling costs and risks. It is incorporated into the usual training process, usually based on stochastic gradient descent (SGD), reducing the user's work, cost and risk in network configuration. When SGD is applied to a wide enough network architecture, they will create hidden features that, depending on the task to be learned, make it very likely that the trainable ANN weight vectors will be initialized very close to the optimal value, which can be quickly reached using state-of-the-art SGD. This can be demonstrated by the smallest possible example of learning a 2-dimensional linear regression using SGD as shown in Figure 3. Although any linear ANN will be functionally equivalent to Figure 3a ), but as Figure 3b ), a non-minimal architecture with an additional wide (e.g., 128 neurons) hidden linear layer will be much more efficient to train using SGD. Training efficiency can be defined by three properties: the small number of training epochs to reach convergence, the short amount of time to train, and the high probability that the network will be trained to a low generalization error when starting from randomly initialized weights.

[0011] An alternative approach to analysis and prediction based on time series data is recurrent models, such as recurrent neural networks (RNNs). In contrast to functional data analysis, recurrent models rely on sampling the time series. Typically, regular sampling is required in order for recurrent models to perform well.

[0012] However, traditional methods, whether FNN or RNN, suffer from poor trainability, especially when limited training data is available or the training data is unlabeled / labeled. This is very common for modeling industrial assets.

[0013] Therefore, there is a need for improved methods, determination systems, and industrial or power systems for generating or updating signal processing logic for processing signals, particularly time series signals, including measurements associated with physical assets. Summary of the invention

[0014] The present disclosure provides a building block for facilitating learning, in particular for situations where data availability is limited, i.e., where the amount and / or quality of training data and / or labels are small. This applies to any learning setting and architecture, whether all / few / no labels are available, whether autoencoders and / or reward labels are used, and whether new adaptive function layers are used in small or large ANNs. In addition, the present disclosure advantageously provides the opportunity to increase the width of the hidden base layer in per-channel processing to achieve a favorable initialization. At the same time, the number of projections generated, i.e., the input width of the remaining (MLP or other) layers, can be strictly controlled. This is very important: if this width is too large, the risk of overfitting is greatly increased. In particular, this may often be the key to achieving successful training when limited training data is available and / or subsequent layers are difficult to train (for example, because these subsequent layers are recurrent layers).

[0015] The present disclosure relates to a method for generating or updating signal processing logic, which is used to process signals, especially time series signals, which include measurement values ​​associated with physical assets. The method includes: training a machine learning ML model, especially a functional neural network FNN, wherein the ML model includes the following operations: receiving a processing signal; determining a plurality of first-level vectors based on the processing signal and a plurality of basis functions, especially by projecting each of the processing signals onto a corresponding basis function in a plurality of basis functions, wherein each vector in the plurality of first-level vectors is related to each projected processing signal in the processing signal through a corresponding basis function; and determining a plurality of second-level vectors based on the plurality of first-level vectors, especially by projecting each vector in the plurality of first-level vectors onto a corresponding transformation matrix in a plurality of transformation matrices, wherein each vector in the plurality of second-level vectors is related to each projected vector in the plurality of first-level vectors through a corresponding transformation matrix.

[0016] In this application, "logic" may refer to a process, procedure, or set of instructions, such as an algorithm, a piece of computer code, or a computer program, that can be executed by a device to control, monitor, and / or analyze a physical asset. For example, logic may be executed on a programmable logic device, such as a field programmable gate array (FPGA), etc.

[0017] In this application, a "physical asset" may refer to a physical unit, such as a power transformer, a distributed energy resource DER unit, or a generator. In this application, a "measurement associated with a physical asset" may refer to an internal and / or external parameter of a physical unit that performs an action and / or generates an input and / or output and / or that can be measured, for example, using a sensor.

[0018] In an embodiment, multiple basis functions are non-trainable.In an embodiment, multiple transformation matrices are trainable.

[0019] In an embodiment, the ML model includes an input layer, and the input layer includes performing the above steps, namely, receiving a processing signal, determining a plurality of first-level vectors, and determining a plurality of second-level vectors.

[0020] In an embodiment, the method further comprises, in particular the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the FNN based on a plurality of second-level vectors.

[0021] In an embodiment, the method further comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the plurality of second-level vectors.

[0022] In an embodiment, the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the plurality of second-level vectors.

[0023] In an embodiment, training the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on multiple second-level vectors.

[0024] In an embodiment, the method further includes providing the trained ML model as at least part of signal processing logic to a device that executes the signal processing logic to control, monitor and / or analyze the physical asset.

[0025] In an embodiment, the method further comprises: receiving a trained ML model; and executing signal processing logic to control, monitor and / or analyze the physical asset.

[0026] In an embodiment, each transformation matrix of the plurality of transformation matrices comprises a plurality of trainable weights.

[0027] In an embodiment, the dimension of at least one of the transformation matrices is different from the dimensions of the remaining transformation matrices, so that the dimension of at least one of the plurality of second-level vectors is different from the dimensions of the remaining second-level vectors.

[0028] In an embodiment, the dimension of the corresponding transformation matrix is ​​determined based on at least one optimization method, in particular integer decision making including grid search and hill climbing.

[0029] In an embodiment, the method further includes determining information related to the health of the physical asset based on the trained ML model, the information including health indicators, a time series of evolution of the health indicators, remaining useful life RUL, failure probability, a time series of evolution of the failure probability, reliability and / or a time series of reliability.

[0030] In an embodiment, determining at least one output comprises processing at least one second level intermediate value through a linear or non-linear function constrained by a plurality of trainable weights.

[0031] In an embodiment, the method further comprises: receiving sensor measurement data captured during operation of the physical asset; and updating the predictive asset health state based on the received sensor measurement data.

[0032] In an embodiment, the physical asset is a power transformer, a distributed energy resource DER unit, or a generator.

[0033] The present disclosure further relates to a method of generating or updating signal processing logic for processing signals, in particular time series signals, which include measurement values ​​associated with physical assets. The method includes training a machine learning ML model, in particular a functional neural network FNN, wherein the ML model includes performing the following operations: receiving a processing signal; for each of the processing signals, determining a first-level vector based on the processing signal and a plurality of basis functions, in particular by projecting each of the processing signals to a corresponding basis function in the plurality of basis functions; and for each first-level vector, determining a second-level vector based on the first-level vector by applying at least one transformation function.

[0034] In an embodiment, the method further comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the second-level vectors.

[0035] In an embodiment, the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the second-level vectors.

[0036] In an embodiment, training the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the second-level vectors.

[0037] In an embodiment, the method further includes providing the trained ML model as at least part of signal processing logic to a device that executes the signal processing logic to control, monitor and / or analyze the physical asset.

[0038] In an embodiment, the method further comprises: receiving a trained ML model; and executing signal processing logic to control, monitor and / or analyze the physical asset.

[0039] In an embodiment, applying the at least one transformation function comprises projecting each vector of the first level vectors to at least one transformation matrix.

[0040] In an embodiment, at least one transformation matrix comprises a plurality of trainable weights.

[0041] In an embodiment, applying at least one transformation function comprises applying a plurality of transformation matrices, and wherein in particular the dimension of at least one of the transformation matrices is different from the dimension of the remaining transformation matrices, so that the dimension of at least one of the second-level vectors is different from the dimension of the remaining second-level vectors.

[0042] In an embodiment, the projection is performed according to the following formula

[0043]

[0044] Among them, z i,j represents the corresponding second-level vector, u i,k denotes the corresponding first-level vector, and Represents the corresponding elements of at least one transformation matrix.

[0045] In an embodiment, i is the index of the signal, j is the index of the matrix element, and k is the index of the basis function.

[0046] In an embodiment, the dimension of the at least one transformation matrix is ​​determined based on at least one optimization method, in particular integer decision making including grid search and hill climbing.

[0047] In an embodiment, a plurality of basis functions are non-trainable.In an embodiment, at least one transformation matrix is ​​trainable.

[0048] In an embodiment, the ML model includes an input layer, and the input layer includes performing the above steps, namely, receiving a processing signal, determining a plurality of first-level vectors, and determining a plurality of second-level vectors.

[0049] In an embodiment, the method further includes: determining information related to the health of the physical asset based on the trained ML model, the information including health indicators, a time series of evolution of health indicators, remaining useful life RUL, failure probability, a time series of evolution of failure probability, reliability and / or a time series of reliability.

[0050] In an embodiment, determining at least one output comprises processing at least one second level intermediate value through a linear or non-linear function constrained by a plurality of trainable weights.

[0051] In an embodiment, the method further comprises: receiving sensor measurement data captured during operation of the physical asset; and updating the predictive asset health state based on the received sensor measurement data.

[0052] In an embodiment, the physical asset is a power transformer, a distributed energy resource DER unit, or a generator.

[0053] The present disclosure also relates to a method of generating or updating signal processing logic for processing signals, particularly time series signals, which include measurement values ​​associated with physical assets, the method comprising providing an ML model trained according to any of the above embodiments as at least part of the signal processing logic to a device that executes the signal processing logic to control, monitor and / or analyze the physical asset.

[0054] The present disclosure further relates to a method for operating and / or maintaining a physical asset, the method comprising: performing a predictive physical asset health analysis on the physical asset using the method of any one of the above embodiments; and automatically performing at least one of the following: scheduling downtime for the physical asset based on the determined predictive physical asset health status; scheduling maintenance work based on the determined predictive physical asset health status; scheduling replacement work based on the determined predictive physical asset health status; and changing maintenance intervals based on the determined predictive physical asset health status.

[0055] The present disclosure further relates to a determination system, which is capable of operating to generate or update signal processing logic, which is used to process signals, especially time series signals, which include measurement values ​​associated with physical assets. The determination system includes a processor, which is configured to: receive the processed signal; and train a machine learning ML model, especially a functional neural network FNN, wherein the ML model includes the following operations: based on the processed signal and a plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function in a plurality of basis functions, determining a plurality of first-level vectors, wherein each vector of the plurality of first-level vectors is related to each projected processed signal in the processed signal through a corresponding basis function; and based on the plurality of first-level vectors, in particular by projecting each vector of the plurality of first-level vectors onto a corresponding transformation matrix in a plurality of transformation matrices, determining a plurality of second-level vectors, wherein each vector of the plurality of second-level vectors is related to each projected vector in the plurality of first-level vectors through a corresponding transformation matrix.

[0056] In an embodiment, the processor is configured, in particular the ML model is configured to: combine the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determine at least one output of the FNN based on multiple second-level vectors.

[0057] In an embodiment, the processor is configured to: combine the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determine at least one output of the ML model, in particular the FNN, based on a plurality of second-level vectors.

[0058] In an embodiment, the processor is configured to cause the ML model to: combine the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determine at least one output of the ML model, in particular the FNN, based on multiple second-level vectors.

[0059] In an embodiment, the ML model is configured to: combine the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determine at least one output of the ML model, in particular the FNN, based on multiple second-level vectors.

[0060] In an embodiment, the processor is configured to train the ML model for: combining second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on multiple second-level vectors.

[0061] In an embodiment, the processor is configured to provide the trained ML model as at least part of signal processing logic to a device that executes the signal processing logic to control, monitor and / or analyze the physical asset.

[0062] In an embodiment, the processor is configured to: receive a trained ML model; and execute signal processing logic to control, monitor and / or analyze a physical asset.

[0063] In an embodiment, each transformation matrix of the plurality of transformation matrices comprises a plurality of trainable weights.

[0064] In an embodiment, the dimension of at least one of the transformation matrices is different from the dimensions of the remaining transformation matrices, so that the dimension of at least one of the plurality of second-level vectors is different from the dimensions of the remaining second-level vectors.

[0065] In an embodiment, the dimension of the corresponding transformation matrix is ​​determined based on at least one optimization method, in particular integer decision making including grid search and hill climbing.

[0066] In an embodiment, the processor is configured to determine information related to the health of the physical asset based on the trained ML model, including health indicators, time series of evolution of health indicators, remaining useful life RUL, failure probability, time series of evolution of failure probability, reliability and / or time series of reliability.

[0067] In an embodiment, determining at least one output comprises processing at least one second level intermediate value through a linear or non-linear function constrained by a plurality of trainable weights.

[0068] In an embodiment, the processor is configured to: receive sensor measurement data captured during operation of the physical asset; and update the predictive asset health state based on the received sensor measurement data.

[0069] In an embodiment, the physical asset is a power transformer, a distributed energy resource DER unit, or a generator.

[0070] The present disclosure also relates to a determination system, which is capable of operating to generate or update signal processing logic, which is used to process signals, especially time series signals, which include measurement values ​​associated with physical assets. The determination system includes a processor, which is configured to: receive processed signals; and train a machine learning ML model, especially a functional neural network FNN, wherein the ML model includes the following operations: for each of the processed signals, based on the processed signal and multiple basis functions, in particular by projecting each of the processed signals onto a corresponding basis function in the multiple basis functions, determine a first-level vector; and for each first-level vector, determine a second-level vector based on the first-level vector by applying at least one transformation function.

[0071] In an embodiment, the processor is configured to perform a method according to any of the embodiments disclosed herein.

[0072] The present disclosure further relates to an industrial or power system comprising: a physical asset; and a determination system according to any of the above embodiments, optionally wherein the determination system is a decentralized controller of the industrial or power system for controlling the asset.

[0073] The method according to any of the embodiments disclosed herein can advantageously monitor and / or estimate quantities of industrial assets, such as operating performance, operating status, or information about external conditions or neighboring systems. One particular quantity that is monitored and / or estimated is the state of health of the industrial asset, which allows understanding the degradation of the asset, to predict the remaining useful life (RUL) of the asset, and to make decisions about operation, maintenance, and repair. The information thus obtained can be used to provide information to human operators, managers, or stakeholders to support their operational or other decisions, or to partially or fully automate the operation of the asset.

[0074] The following list of aspects provides alternative and / or additional features of the present disclosure:

[0075] 1. A method of generating or updating signal processing logic for processing signals, particularly time series signals, comprising measurements associated with a physical asset, the method comprising:

[0076] Training a machine learning ML model, in particular a functional neural network FNN, wherein the ML model includes performing the following operations:

[0077] receiving and processing signals;

[0078] determining a plurality of first-order vectors based on the processed signal and the plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function of the plurality of basis functions,

[0079] wherein each of the plurality of first-order vectors is associated with each of the projected processed signals in the processed signals via a corresponding basis function; and

[0080] determining a plurality of second-level vectors based on the plurality of first-level vectors, in particular by projecting each vector of the plurality of first-level vectors onto a corresponding transformation matrix of the plurality of transformation matrices,

[0081] Each of the plurality of second-level vectors is associated with each of the projected vectors of the plurality of first-level vectors via a corresponding transformation matrix.

[0082] 2. The method according to aspect 1, further comprising:

[0083] combining the second level vectors, in particular by concatenating the second level vectors into a concatenated vector; and / or

[0084] At least one output of the FNN is determined based on the plurality of second level vectors.

[0085] 3. The method of claim 1 or 2, providing the trained ML model as at least part of signal processing logic to a device that executes signal processing logic to control, monitor and / or analyze a physical asset.

[0086] 4. The method according to aspect 3 further comprises:

[0087] receiving a trained ML model; and

[0088] Execute signal processing logic to control, monitor, and / or analyze physical assets.

[0089] 5. A method as described in any one of aspects 1 to 4, wherein each transformation matrix of the plurality of transformation matrices comprises a plurality of trainable weights.

[0090] 6. A method as described in any one of Aspects 1 to 5, wherein the dimension of at least one of the transformation matrices is different from the dimensions of the remaining transformation matrices, so that the dimension of at least one of the multiple second-level vectors is different from the dimensions of the remaining second-level vectors.

[0091] 7. A method as described in any one of aspects 1 to 6, wherein the dimension of the corresponding transformation matrix is ​​determined based on at least one optimization method, in particular integer decision including grid search and hill climbing.

[0092] 8. The method as described in any one of Aspects 1 to 7 comprises determining information related to the health of the physical asset based on a trained ML model, the information comprising health indicators, a time series of the evolution of the health indicators, a remaining useful life RUL, a failure probability, a time series of the evolution of the failure probability, reliability and / or a time series of reliability.

[0093] 9. A method as described in any of aspects 1 to 8, wherein determining at least one output includes processing at least one second-level intermediate value through a linear or non-linear function constrained by a plurality of trainable weights.

[0094] 10. The method according to any one of aspects 1 to 9, further comprising:

[0095] receiving sensor measurement data captured during operation of the physical asset; and

[0096] Based on the received sensor measurement data, the predictive asset health status is updated.

[0097] 11. The method of any one of aspects 1 to 10, wherein the physical asset is a power transformer, a distributed energy resource DER unit, or a generator.

[0098] 12. A method for generating or updating signal processing logic for processing signals, particularly time series signals, comprising measurements associated with a physical asset, the method comprising providing an ML model trained according to any of the preceding aspects as at least part of the signal processing logic to a device executing the signal processing logic to control, monitor and / or analyze the physical asset.

[0099] 13. A method of operating and / or maintaining a physical asset, the method comprising:

[0100] Performing a predictive physical asset health analysis on a physical asset using the method of any of the preceding aspects; and

[0101] Automatically perform at least one of the following: schedule downtime of a physical asset based on a determined predictive physical asset health state; schedule maintenance work based on a determined predictive physical asset health state; schedule replacement work based on a determined predictive physical asset health state; change a maintenance interval based on a determined predictive physical asset health state.

[0102] 14. A determination system operable to generate or update signal processing logic for processing signals, in particular time series signals, comprising measurements associated with a physical asset, the determination system comprising a processor configured to:

[0103] receiving and processing a signal; and

[0104] Training a machine learning ML model, specifically a functional neural network FNN, where the ML model includes the following operations:

[0105] determining a plurality of first-order vectors based on the processed signal and the plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function of the plurality of basis functions,

[0106] wherein each of the plurality of first-order vectors is associated with each of the projected processed signals in the processed signals via a corresponding basis function; and

[0107] determining a plurality of second-level vectors based on the plurality of first-level vectors, in particular by projecting each vector of the plurality of first-level vectors onto a corresponding transformation matrix of the plurality of transformation matrices,

[0108] Each of the plurality of second-level vectors is associated with each of the projected vectors of the plurality of first-level vectors via a corresponding transformation matrix.

[0109] 15. An industrial or power system comprising:

[0110] physical assets; and

[0111] A determination system as described in aspect 14, optionally wherein the determination system is a decentralized controller of an industrial or power system for controlling assets.

[0112] Various exemplary embodiments of the present disclosure are intended to provide features that will become apparent by reference to the following description when taken in conjunction with the accompanying drawings. According to various embodiments, exemplary systems, methods, and devices are disclosed herein. However, it should be understood that these embodiments are presented by way of example and not limitation, and it will be apparent to those of ordinary skill in the art reading this disclosure that various modifications may be made to the disclosed embodiments while still within the scope of this disclosure.

[0113] Therefore, the present disclosure is not limited to the exemplary embodiments and applications described and illustrated herein. In addition, the specific order and / or hierarchy of steps in the methods disclosed herein are merely exemplary methods. Based on design preferences, the specific order or hierarchy of steps of the disclosed methods or processes can be rearranged while still within the scope of the present disclosure. Therefore, it will be understood by those of ordinary skill in the art that the methods and techniques disclosed herein present various steps or actions in an example order, and unless otherwise expressly stated, the present disclosure is not limited to the specific order or hierarchy presented.

[0114] In the following, exemplary embodiments of the present disclosure will be described. Note that, unless otherwise stated or apparent, some aspects of any of the described embodiments may also be found in some other embodiments. However, in order to improve understandability, each aspect will be described in detail only when first mentioned, and any repeated description of the same aspect will be omitted.

[0115] The above and other aspects and embodiments thereof are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] Figure 1a )and Figure 1b ) illustrates the functional neural network architecture.

[0117] Figure 2 Illustration of a functional neural network architecture.

[0118] Figure 3a )and Figure 3b ) illustrates a minimal architecture and a non-minimal architecture.

[0119] Figure 4a )and Figure 4b ) illustrates a flowchart of a method according to an embodiment of the present disclosure.

[0120] Figure 5A functional neural network architecture according to an embodiment of the present disclosure is illustrated.

[0121] Figure 6 The generalized structure of a functional neural network architecture according to an embodiment of the present disclosure is illustrated.

[0122] Figure 7a )and Figure 7b ) illustrates a performance comparison between functional neural network architectures, including a functional neural network architecture according to an embodiment of the present disclosure.

[0123] Figure 8a )and Figure 8b ) illustrates a performance comparison between functional neural network architectures, including a functional neural network architecture according to an embodiment of the present disclosure.

[0124] Figure 9a )and Figure 9b ) illustrates devices and systems according to embodiments of the present disclosure. DETAILED DESCRIPTION

[0125] Figure 4a ) illustrates a flow chart of a method according to an embodiment of the present disclosure. In particular, the method according to an embodiment of the present disclosure includes training a machine learning ML model, in particular a functional neural network FNN, and Figure 4a ) illustrates a method performed by an ML model. At S401, a processed signal is received. The processed signal may be a time series signal including measurement values ​​associated with at least one physical asset. The processed signal may optionally be pre-processed before being received, in particular by the ML model (e.g. scaling, data cleaning, etc.). At S402, a plurality of first-level vectors are determined based on the processed signal and a plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function in a plurality of basis functions, wherein each of the plurality of first-level vectors is associated with each of the projected processed signals in the processed signal via a corresponding basis function. At S403, a plurality of second-level vectors are determined based on the plurality of first-level vectors, in particular by projecting each of the plurality of first-level vectors onto a corresponding transformation matrix in a plurality of transformation matrices, wherein each of the plurality of second-level vectors is associated with each of the projected vectors in the plurality of first-level vectors via a corresponding transformation matrix.

[0126] Figure 4b ) illustrates a flow chart of a method according to an embodiment of the present disclosure. In particular, the method according to an embodiment of the present disclosure includes training a machine learning ML model, in particular a functional neural network FNN, and Figure 4b) illustrates a method performed by an ML model. At S401', a processed signal is received. The processed signal may be a time series signal including a measurement value associated with at least one physical asset. The processed signal may optionally be preprocessed before being received, in particular by the ML model (e.g. scaling, data cleaning, etc.). At S402', for each of the processed signals, a first-level vector is determined based on the processed signal and a plurality of basis functions, in particular by projecting each processed signal onto a corresponding basis function in the plurality of basis functions. At S403', for each first-level vector, a second-level vector is determined based on the first-level vector by applying at least one transformation function.

[0127] Figure 5 The functional neural network architecture according to an embodiment of the present disclosure is illustrated. The functional neural network of the present disclosure is hereinafter referred to as an adaptive equivalent FNN (AEFNN) layer. The basic linear AEFNN architecture is Figure 5 and includes the following operations:

[0128] Project the channel (S510) onto a number of fixed basis functions This step does not involve trainable weights, so it can be done outside of SGD training. That is, the fixed basis function It is not trainable.

[0129] Optionally, the first projection u of the input to the ANN i，k The output of 511 is normalized (e.g., centered and scaled to unit standard deviation). Subsequently, each channel i has a trainable regular ANN layer with linear activation. These trainable layers act on these channels in parallel.

[0130] ·The first projection u i，k The (optionally normalized) output of 511 is projected (S520) onto the weight matrix A i The corresponding weight The weight matrix A for each data channel i i It is trainable.

[0131] Sum the results of the second projection S520 over k to determine the second projection z i，j Output of 521.

[0132] Optionally, depending on how the output of the second projection is subsequently used, the second projection z i，j The outputs of 521 are arranged (S530). For example, if the adaptive equivalent FNN layer is followed by more ANN layers, their outputs may be concatenated, shaped, stacked, etc.

[0133] Second projection z i，jThe output is the function input f i (t) Projection onto basis functions, which are fixed basis functions A linear combination of. Since the weight matrix A for each data channel i i is trainable, so the fixed basis function The linear combination of Figure 2 Adaptive basis functions This relationship is shown in the mathematical derivation, and the basis functions effectively used are circled in 522.

[0134] The input sample can be referred to as the processed signal. i，k The output of can refer to multiple first-level vectors. The second projection z i，j The output of can refer to multiple second-level vectors. i May refer to a transformation matrix.

[0135] In an embodiment, the method further comprises, in particular the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the FNN based on a plurality of second-level vectors.

[0136] In an embodiment, the method further comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the plurality of second-level vectors.

[0137] In an embodiment, the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on the plurality of second-level vectors.

[0138] In an embodiment, training the ML model comprises performing the following operations: combining the second-level vectors, in particular by concatenating the second-level vectors into a concatenated vector; and / or determining at least one output of the ML model, in particular the FNN, based on multiple second-level vectors.

[0139] In an embodiment, the method further includes providing the trained ML model as at least part of signal processing logic to a device that executes the signal processing logic to control, monitor and / or analyze the physical asset.

[0140] In an embodiment, the method further comprises: receiving a trained ML model; and executing signal processing logic to control, monitor and / or analyze the physical asset.

[0141] In an embodiment, each transformation matrix of the plurality of transformation matrices comprises a plurality of trainable weights.

[0142] In an embodiment, the dimension of at least one of the transformation matrices is different from the dimensions of the remaining transformation matrices, so that the dimension of at least one of the plurality of second-level vectors is different from the dimensions of the remaining second-level vectors.

[0143] In an embodiment, the dimension of the corresponding transformation matrix is ​​determined based on at least one optimization method, in particular integer decision making including grid search and hill climbing.

[0144] In an embodiment, the method further includes determining information related to the health of the physical asset based on the trained ML model, the information including health indicators, a time series of evolution of the health indicators, remaining useful life RUL, failure probability, a time series of evolution of the failure probability, reliability and / or a time series of reliability.

[0145] In an embodiment, determining at least one output comprises processing at least one second level intermediate value through a linear or non-linear function constrained by a plurality of trainable weights.

[0146] In an embodiment, the method further comprises: receiving sensor measurement data captured during operation of the physical asset; and updating the predictive asset health state based on the received sensor measurement data.

[0147] In an embodiment, the physical asset is a power transformer, a distributed energy resource DER unit, or a generator.

[0148] In an embodiment, the method further comprises providing the ML model trained according to any of the above embodiments as at least part of signal processing logic to a device executing signal processing logic to control, monitor and / or analyze a physical asset.

[0149] In an embodiment, the method further includes: performing a predictive physical asset health analysis on the physical asset using the method of any one of the above embodiments; and automatically performing at least one of the following: scheduling downtime of the physical asset based on the determined predictive physical asset health status; scheduling maintenance work based on the determined predictive physical asset health status; scheduling replacement work based on the determined predictive physical asset health status; changing the maintenance interval based on the determined predictive physical asset health status.

[0150] The complete overall design of the adaptive equivalent FNN layer is Figure 6 As shown in Figure 5 The only difference to the linear layers described above is the generalization of per-channel layers, which may be non-linear and may include multiple base layers. Thus, this involves performing the following operations:

[0151] Project the channel (S610) onto many fixed basis functions This step does not involve trainable weights, so it can be done outside of SGD training. That is, the fixed basis function It is not trainable.

[0152] Optionally, the first projection u of the input to the ANN i，k The output of is normalized (e.g., centered and scaled to unit standard deviation). Subsequently, each channel i has one or more trainable conventional ANN layers (fully connected or otherwise) with linear or nonlinear activations. These trainable layers act on these channels in parallel.

[0153] ·The first projection u i，k The (optionally normalized) output of is projected (S620) to the weight matrix A of the first trainable regular ANN layer among the N trainable regular ANN layers i The corresponding weight The weight matrix A for each data channel i i is trainable. When N is greater than one, the output of the previous projection is then projected, i.e., the projection u i，k To the weight matrix A of the first trainable regular ANN layer i The result is projected into the weight matrix A of the second trainable conventional ANN layer i And so on.

[0154] Sum the results of the second projection S620 over k to determine the second projection z i，j Output of 621.

[0155] Optionally, depending on how the output of the second projection is subsequently used, the second projection z i，j The outputs of 621 are arranged (S630). For example, if the adaptive equivalent FNN layer is followed by more ANN layers, their outputs may be concatenated, shaped, stacked, etc.

[0156] Note that as mentioned in the above steps, the term adaptive equivalent FNN layer refers to a construction which technically can be a sequence of multiple non-trainable and trainable layers and other matrix manipulation operations (concatenation, reshaping, stacking, etc.).

[0157] The adaptive equivalent FNN layer is usually only a part of the entire ANN. For example, in Figures 3 and 4, there are the remaining MLP networks S530 and S630 after the adaptive equivalent layer. In general, the adaptive equivalent FNN layer is usually the first layer of the ANN, followed by any other ANN layers.

[0158] In an embodiment, function layers including at least one adaptive equivalent function layer are stacked, in particular by performing the following operations: applying the function layer to function input data; interpreting the numerical output vector from the function layer as function data (e.g., sampled function evaluations), possibly after rearranging the vector (reshaping, transposing, etc.); and applying a second function layer to the data produced by the previous operation.

[0159] In an embodiment, the processed signal comprises at least two variables. In an embodiment, a basis function is selected for a respective variable of the at least two variables. In an embodiment, the processed signal is projected onto the basis function selected for the respective variable of the at least two variables.

[0160] In an embodiment, the ML is or is part of a trainable neural network system that, after training, is used to process data from industrial assets according to any of the above embodiments. In an embodiment, the data processing is performed offline or online, where the term "online" refers to the trained neural network system continuously evaluating new data recorded from the asset(s) or evaluating it at regular or irregular time intervals.

[0161] Figures 7 and 8 illustrate the performance of a simple RUL prediction task on the NASA Turbofan Data Set. Specifically, an ANN for predicting RUL labels was established and trained on the first dataset FD001. The training dataset was randomly divided into 80% training data and 20% validation data, and the training was repeated multiple times to obtain the statistical distribution of validation loss and training time. Two different sets of input channels were used: (a) all available input channels in the dataset, and (b) 6 input channels pre-selected according to their importance for predicting RUL.

[0162] Six architectures were compared. Three of them learn to predict RUL based on data taken from a fixed sliding window of length 21 samples, and three combine the sliding window input with past information via a recurrent neural network (RNN) architecture. For the RNN, we used a gated recurrent unit (GRU) network. We also compared a simplified RNN that processes temporal information with two parallel processing streams consisting of a temporal convolutional network and a simple cumulative sum.

[0163] Figure 7 compares the validation loss of different architectures for the RUL prediction task for two sets of input channels. In particular, Figure 7a ) illustrates the result when all input channels are used, and Figure 7b) illustrates the results when using 6 selected input channels. It can be seen that the network with a recurrent architecture performs better than the network without a recurrent architecture. Comparing the AE FNN architecture that does not directly refer to the FNN with a feature function dedicated to this task, it is observed that the performance is similar, so the AE FNN can successfully learn to predict RUL without a dedicated feature function. In contrast, the performance of AdaFNN is not as robust. Compared to RNN (GRU network), AE FNN is able to compete with it.

[0164] The training time of different architectures is compared in Figure 8. In particular, Figure 8a ) illustrates the result when all input channels are used, and Figure 8b ) shows the results when using 6 selected input channels. It can be seen that the AEFNN without recurrence trains fastest, even faster than the FNN with a dedicated feature function. In addition, an AE FNN variant is the fastest when combined with an RNN. This suggests that the AE FNN layer is a strong architectural choice and may have favorable scaling properties for functional input data.

[0165] Figure 9a ) illustrates a determination system 910 that is operable to generate or update signal processing logic for processing signals, particularly time series signals, which include measurement values ​​associated with a physical asset 920, the determination system 910 comprising a processor 911 configured to: receive the processed signal; and train a machine learning ML model, particularly a functional neural network FNN, wherein the ML model comprises performing the following operations: determining a plurality of first-level vectors based on the processed signal and a plurality of basis functions, particularly by projecting each of the processed signals onto a corresponding basis function in a plurality of basis functions, wherein each of the plurality of first-level vectors is related to each projected processed signal in the processed signal via a corresponding basis function; and determining a plurality of second-level vectors based on the plurality of first-level vectors, particularly by projecting each of the plurality of first-level vectors onto a corresponding transformation matrix in a plurality of transformation matrices, wherein each of the plurality of second-level vectors is related to each projected vector in the plurality of first-level vectors via a corresponding transformation matrix.

[0166] In an embodiment, the processor 911 is further configured to: receive processed signals; and train a machine learning ML model, in particular a functional neural network FNN, wherein the ML model includes performing the following operations: for each of the processed signals, determining a first-level vector based on the processed signal and a plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function in a plurality of basis functions; and for each first-level vector, determining a second-level vector based on the first-level vector by applying at least one transformation function.

[0167] In an embodiment, the processor 911 is configured to transmit / provide / transmit the received processed signal to the ML model. In an embodiment, the ML model comprises executing receiving the processed signal, in particular receiving the processed signal from the processor.

[0168] In an embodiment, the processor 911 is configured to perform a method according to any one of the above embodiments.

[0169] Figure 9b ) illustrates an industrial or power system 900, which includes: a physical asset 920; and a determination system 910 according to any of the above-mentioned embodiments, optionally, wherein the determination system 910 is a distributed controller of the industrial or power system 900 for controlling the asset.

[0170] Although various embodiments of the present disclosure have been described above, it should be understood that these embodiments are presented only by way of example rather than by way of limitation. Similarly, various figures may depict example architectures or configurations, which are provided to enable those of ordinary skill in the art to understand the exemplary features and functions of the present disclosure. However, those of ordinary skill in the art will appreciate that the present disclosure is not limited to the illustrated example architectures or configurations, but may be implemented using various alternative architectures and configurations. In addition, as will be appreciated by those of ordinary skill in the art, one or more features of an embodiment may be combined with one or more features of another embodiment described herein. Therefore, the breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments.

[0171] It should also be understood that any reference to an element using names such as "first," "second," etc. herein does not generally limit the number or order of those elements. Rather, these names may be used herein as a convenient way to distinguish between two or more elements or element instances. Thus, reference to a first and a second element does not mean that only two elements can be used, or that the first element must precede the second element in some manner.

[0172] In addition, those of ordinary skill in the art will appreciate that information and signals may be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, and symbols (such as may be mentioned in the above description) may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0173] Those of skill will further understand that any of the various illustrative logical blocks, units, processors, devices, circuits, methods, and functions described in conjunction with the aspects disclosed herein may be implemented by electronic hardware (e.g., digital implementations, analog implementations, or a combination of the two), firmware, various forms of programs or design codes containing instructions (which, for convenience, may be referred to herein as "software" or "software units"), or any combination of these technologies.

[0174] In order to clearly illustrate this interchangeability of hardware, firmware and software, various illustrative components, blocks, units, circuits and steps have been described above generally in terms of their functions. Whether such functions are implemented as hardware, firmware or software, or the combination of these technologies depends on the specific application and design constraints imposed on the overall system. The technician can implement the described functions in various ways for each specific application, but such implementation decision will not lead to deviation from the scope of this disclosure. According to various embodiments, processors, devices, components, circuits, structures, machines, units, etc. can be configured to perform one or more functions described herein. The term "configured to" or "configured for" used herein with respect to a specified operation or function refers to a processor, device, component, circuit, structure, machine, unit, etc. that is physically constructed, programmed and / or arranged to perform a specified operation or function.

[0175] In addition, the technician will understand that the various illustrative methods, logic blocks, units, devices, components and circuits described herein can be implemented or performed in an integrated circuit (IC) that can include a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device or any combination thereof. The logic blocks, units and circuits may further include antennas and / or transceivers to communicate with various components within the network or within the device. The general-purpose processor can be a microprocessor, but in an alternative, the processor can be any traditional processor, controller or state machine. The processor can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other suitable configuration to perform the functions described herein. If implemented in software, these functions can be stored as one or more instructions or codes on a computer-readable medium. Therefore, the steps of the method or algorithm disclosed herein can be implemented as software stored on a computer-readable medium.

[0176] Computer-readable media include both computer storage media and communication media, including any medium capable of transferring a computer program or code from one place to another. Storage media can be any available medium that a computer can access. By way of example and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0177] In addition, memory or other storage devices and communication components can be used in embodiments of the present disclosure. It should be appreciated that, for the sake of clarity, the above description has described embodiments of the present disclosure with reference to different functional units and processors. However, it is apparent that any suitable functional distribution between different functional units, processing logic elements or domains can be used without departing from the present disclosure. For example, the functions illustrated as being performed by a separate processing logic element or controller can be performed by the same processing logic element or controller. Therefore, references to specific functional units are only references to suitable devices for providing the described functions, rather than representing a strict logical or physical structure or organization.

[0178] Various modifications to the embodiments described in this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the embodiments shown herein, but is intended to be consistent with the maximum scope consistent with the novel features and principles disclosed in the appended claims.

Claims

1. A method of generating or updating signal processing logic for processing a signal, in particular a time series signal, comprising: Training a machine learning ML model, in particular a functional neural network FNN, wherein the ML model comprises performing the following operations: receiving the processed signal; determining, for each of the processed signals, a first level vector based on the processed signal and a plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function of the plurality of basis functions; as well as For each first level vector, a second level vector is determined based on the first level vector by applying at least one transformation function.

2. The method of claim 1, further comprising: combining said second-level vectors, in particular by concatenating said second-level vectors into a concatenated vector; and / or Based on the second-level vector, at least one output of the ML model, in particular the FNN, is determined.

3. The method of claim 1 , providing the trained ML model as at least part of signal processing logic to a device that executes the signal processing logic to control, monitor and / or analyze the physical asset.

4. The method of claim 3, further comprising: receiving the trained ML model; as well as The signal processing logic is executed to control, monitor and / or analyze the physical asset.

5. The method according to any one of claims 1 to 4, wherein: Applying at least one transformation function includes projecting each of the first-level vectors to at least one transformation matrix.

6. The method of claim 5, wherein: The at least one transformation matrix includes a plurality of trainable weights.

7. The method according to claim 5 or 6, wherein: Applying at least one transformation function comprises applying a plurality of transformation matrices, and wherein in particular the dimension of at least one of the transformation matrices is different from the dimensions of the remaining transformation matrices, so that the dimension of at least one of the second-level vectors is different from the dimensions of the remaining second-level vectors.

8. The method according to any one of claims 5 to 7, wherein the projection is based on Executed, among which, z i,j represents the corresponding second-level vector, u i,k denotes the corresponding first-level vector, and represents the corresponding elements of the at least one transformation matrix.

9. The method according to any one of claims 5 to 8, wherein: The dimension of the at least one transformation matrix is ​​determined based on at least one optimization method, in particular based on integer decision making including grid search and hill climbing.

10. The method of any one of claims 1 to 9, comprising determining information related to the health of the physical asset based on the trained ML model, the information comprising health indicators, a time series of evolution of health indicators, remaining useful life RUL, failure probability, a time series of evolution of failure probability, reliability and / or a time series of reliability.

11. The method according to any one of claims 1 to 10, wherein: Determining the at least one output includes processing at least one second level intermediate value through a linear or nonlinear function constrained by the plurality of trainable weights.

12. The method of any one of claims 1 to 11, further comprising: receiving sensor measurement data captured during operation of the physical asset; as well as Based on the received sensor measurement data, the predictive asset health status is updated.

13. The method according to any one of claims 1 to 12, wherein: The physical asset is a power transformer, a distributed energy resource DER unit, or a generator.

14. A method of generating or updating signal processing logic for processing signals, in particular time series signals, comprising providing an ML model trained according to any of the preceding claims as at least part of the signal processing logic to a device executing the signal processing logic to control, monitor and / or analyze the physical asset.

15. A method of operating and / or maintaining a physical asset, the method comprising: performing a predictive physical asset health analysis on the physical asset using the method of any of the preceding claims; as well as Automatically perform at least one of the following: schedule downtime of the physical asset based on the determined predictive physical asset health state; schedule maintenance work based on the determined predictive physical asset health state; schedule replacement work based on the determined predictive physical asset health state; change maintenance intervals based on the determined predictive physical asset health state.

16. A determination system operable to generate or update signal processing logic for processing signals, in particular time series signals, comprising measurements associated with a physical asset, the determination system comprising a processor configured to: receiving the processed signal; and Training a machine learning ML model, in particular a functional neural network FNN, wherein the ML model comprises performing the following operations: for each of the processed signals, determining a first level vector based on the processed signal and a plurality of basis functions, in particular by projecting each of the processed signals onto a corresponding basis function of the plurality of basis functions; and For each first level vector, a second level vector is determined based on the first level vector by applying at least one transformation function.

17. An industrial or power system comprising: Physical assets; as well as The determination system of claim 16, optionally wherein the determination system is a decentralized controller for an industrial or power system for controlling assets.

Images