Industrial equipment anomaly detection method based on reliable inference attention nerve process

Abstract

Aiming at the problem that an existing data-driven model cannot effectively describe high-dimensional data features of modern complex industrial equipment due to simple Gaussian distribution hypothesis, the invention provides an industrial equipment anomaly detection method based on a reliable attention nerve inference process, and the industrial equipment anomaly detection method is used for learning and inferring real distribution of high-dimensional data of the complex industrial equipment. The method comprises the following steps: firstly, aiming at a sampling hole problem caused by unlearnable prior distribution, enabling a model to obtain flexible and learnable prior distribution by using a normalized stream; secondly, smooth transition of posterior distribution is realized by using a Shenchang differential equation, so that priori distribution and posterior distribution can be perfectly matched; finally, the effectiveness and flexibility of the method for detecting the abnormity of the industrial equipment in a real industrial scene are verified through an industrial robot abnormity detection (state evaluation) experiment and a numerical control machine tool ball screw abnormity detection (state evaluation) experiment.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial equipment detection, and specifically relates to an industrial equipment anomaly detection method based on a reliable inference attention neural process. Background Art

[0002] Many industrial equipment problems involve evaluating the working state of an industrial equipment production process through data collection in an industrial environment, which is also known as anomaly detection. Typical examples include the detection of early faults, isolation, origin, and identification of specific fault types (fault diagnosis), as well as prediction of remaining useful life (fault prediction), etc. Modern industrial equipment often has complex structural principles and operating conditions, resulting in diverse data characteristics of its measured variables. These different data characteristics in industrial equipment often cannot be accurately solved by simple mathematical modeling. Therefore, in most cases, data-driven methods are the only suitable choice. Currently, many studies have proposed various data-driven methods for different types of industrial equipment anomaly detection tasks and have achieved remarkable success in many typical industrial equipment anomaly detection scenarios.

[0003] Among them, the research on supervised learning strategies is the most extensive. In this field, generative models are often contrasted with discriminative models. Generative models focus on understanding the underlying data distribution, and their goal is to model the distribution of a single class, while discriminative models aim to map input data to appropriate labels, and their focus is on modeling the decision boundary between classes. However, like most industrial scenarios, observed data is often difficult to label, and only positive sample data can be collected. And discriminative models require each training sample to have a label, which is a restrictive requirement for industrial equipment anomaly detection tasks because labels may be expensive or even impossible to collect. In reality, we also hope to learn some or all of its features from unlabeled data. This leads us to unsupervised learning, and the capabilities and performance of specific unsupervised learning models are crucial for industrial equipment anomaly detection tasks. With the development of deep learning, unsupervised learning models based on generative models, such as generative adversarial networks (GANs), variational autoencoders (VAEs), and neural processes (NPs), have been widely applied to industrial equipment anomaly detection tasks.

[0004] GAN realizes industrial equipment anomaly detection by adversarially training the generator and discriminator. However, there is no explicit expression for the marginal likelihood probability and posterior probability distribution in GAN. Therefore, it is impossible to quantify the uncertainty of the GAN model for equipment anomaly detection in a real industrial environment, and the model is unstable, prone to mode collapse when dealing with high-dimensional data of industrial equipment. VAE first encodes high-dimensional original observation data into a low-dimensional latent variable space, and then tries to reproduce the real observation data by sampling information from the latent variable space through a decoder. Its decoding process is also called the generation process. From an engineering perspective, VAE can model the potential space distribution of data. Although it can capture complex data distributions, it mainly focuses on the data generation process, cannot directly quantify the uncertainty of predictions, and needs to be retrained to adapt to new observation data, unable to meet the time efficiency requirements of industrial equipment anomaly detection where abnormal samples are extremely scarce. In addition, many of these methods only focus on identifying the static working modes of processes based on fixed-length timestamps, while ignoring dynamic change patterns and subtle degradations hidden under instantaneous or long-range time information. In contrast, some studies have tried to address complex change problems by incorporating machine learning models into a stochastic process framework.

[0005] To address this issue, a pioneering architecture called NPs integrates the powerful non-linear modeling capabilities of deep neural networks into the mathematical framework of stochastic processes. In addition, it adopts an efficient optimization scheme of variational inference, which can model any complex distribution on continuous functions without being restricted by any kernel function assumptions. This design enables NPs to predict the output and its uncertainty of new data points by considering the existing observed data (referred to as context), thus naturally adapting to new data without having to retrain the model from scratch.

[0006] However, traditional NPs assume that their prior and posterior distributions are simple Gaussian distributions, and they are unable to model the complex data distributions in real industrial scenarios. Therefore, this assumption greatly weakens the ability of NPs in industrial equipment anomaly detection tasks. Distribution transformation methods have been widely studied to improve the accuracy of the generative model in fitting data distributions, such as coupling flows, autoregressive flows, inverse autoregressive flows, planar flows, and neural ordinary differential equations (NODEs). Planar flows provide a direct calculation method for the Jacobian determinant, making their calculation efficiency higher. When modeling complex distributions, they do not introduce excessive computational burdens. Neural ordinary differential equations use ODE solvers for forward propagation and do not require explicit calculation of the Jacobian matrix for the entire distribution transformation process. Therefore, the computational burden brought by NODEs when modeling complex posterior distributions can be almost negligible. Coupling flows only change a part each time, which improves the calculation efficiency of the Jacobian determinant. However, when modeling complex distributions in industrial scenarios, it will cause the model to become large and slow. Autoregressive flows and inverse autoregressive flows rely on previous dimensions when dealing with the Jacobian determinant of each dimension, making their calculation of the Jacobian matrix complex and intensive. In some existing studies, it is proposed to use NODEs to smoothly transition the posterior distribution of VAE to a complex distribution, and to use real-valued non-volume-preserving to achieve the transformation between the posterior distribution of CVAE and a complex real distribution. However, most of the existing distribution transformation works only parameterize their posterior distributions as multivariate Gaussian distributions, or use distribution transformation methods to transform them into complex distributions, and then calculate the KL divergence between the prior in the form of a simple Gaussian distribution and them. Obviously, this cannot achieve the adaptation of the model's prior distribution and posterior distribution because you cannot use a simple Gaussian distribution to fit a complex unknown distribution. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide an industrial equipment anomaly detection method based on a reliable inference attention neural process, aiming to achieve an accurate characterization of complex data distributions in real industrial scenarios.

[0008] To achieve the above purpose, the present invention provides the following technical solutions:

[0009] An industrial equipment anomaly detection method based on a reliable inference attention neural process, comprising the following steps:

[0010] Step 1: For a given dataset of industrial equipment Define the joint distribution and construct a latent variable generative model based on the attention neural process;

[0011] Step 2: Divide the dataset into a context dataset C and a target dataset T, and use amortized variational inference to learn the variational posterior of the latent variables to obtain the evidence lower bound of the attention neural process.

[0012] Step 3: Input the context dataset C into the deterministic path encoder of the latent variable generation model to obtain a prior distribution in the form of a simple Gaussian distribution; introduce normalizing flow, and convert the prior distribution into a learnable multimodal prior distribution through reversible transformation;

[0013] Step 4: Input the target dataset T into the latent layer path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution; use neural ordinary differential equations to perform continuous transformation on the deterministic Gaussian distribution, and solve through an ODE solver to obtain the target complex distribution that fits the true posterior;

[0014] Step 5: Construct an improved evidence lower bound containing the multimodal prior term and the target complex distribution that fits the true posterior as the loss function, and train the model through the reparameterization trick;

[0015] Step 6: Use the trained model to calculate the conditional probability density of new observed data, and achieve device anomaly detection through threshold judgment.

[0016] Furthermore, in the above Step 1, the dataset of the given industrial equipment is expressed as:

[0017]

[0018] where: x i is the observed data point; represents the event space of all observed data points; Ω represents the sampling space; t i is the time; n is the number of data points;

[0019] The joint distribution is defined as:

[0020]

[0021] where: represents the joint distribution of the observed data point x i ; p represents the abstract probability distribution of the observed data point x i ; p(f) represents the marginal probability density function of the observed data point x i ; p(x i |f,t i ) represents the conditional probability density function of the observed data point x i under the condition of the given transformation f and time t i ; x i represents the observed data point; t i represents the set of all time points; f represents the transformation;

[0022] Use the attention neural process to represent random events Suppose It can be parameterized by a high-dimensional random vector z and add observation noise to replace X i = F(t i ); Express F(t) = g(t, z) as a certain fixed learnable function g to obtain a latent variable generation model:

[0023]

[0024] where: p(z|t i , x i ) represents the posterior probability distribution of the latent variable z given all observation time points t i and its corresponding observation data x i . p(z) is the prior probability distribution; is the probability density of the learnable function g generating the observation data point x i after adding noise; t i represents the set of all time points, that is, the time t i corresponding to each observation data x i .

[0025] Furthermore, in the second step, the evidence lower bound of the attention neural process is expressed as:

[0026]

[0027] where: p(x m+1:n |t 1:n , x 1:m ) represents the conditional probability of predicting the target data set x m+1:n , t m+1:n )} given the context data set {(x m+1:n ); represents the expectation of the true posterior distribution q(z|t 1:n , x 1:n ); p(x i |z, t i ) represents the likelihood given the latent variable z and the time point t i ; D kl (·) represents the Kl divergence measuring the difference between two probability distributions; q(z|t 1:m , x 1:m ) represents the distribution using the variational posterior to replace the prior; q(z|t 1:n , x 1:n ) represents the true posterior distribution; C = {(x i , t i ), i ∈ [1, m]}, that is, C = {(x 1:m , t 1:m )} represents the context data set; T = {(xi ,t i ) where \(i\in[1,n]\), i.e., \(T = \{(x m+1:n ,t m+1:n )\}\) represents the target data set.

[0028] Furthermore, in the third step, the obtained multimodal prior distribution is expressed as:

[0029]

[0030] where: \(q c (z'|x)\) represents the multimodal prior distribution; \(q c (z|x)\) is a prior distribution in the form of a simple Gaussian distribution; \(z\) is a random variable.

[0031] Furthermore, by introducing normalizing flows, the method of converting the prior distribution into a learnable multimodal prior distribution through a reversible transformation is:

[0032] Input the context data set \(C = \{(x 1:m ,t 1:m )\}\) into the deterministic path encoder of the latent variable generation model to obtain the prior distribution \(q c (z|x)\) in the form of a simple Gaussian distribution; sample a random variable \(z_0\) from it, introduce normalizing flows, and use a transformation group to perform shape transformation:

[0033] f(z)=z_0 + uh(w T z_0 + b)

[0034] where: \(f(z)\) represents the new variable obtained by applying a set of transformations; \(z_0\) is a sampled random variable; \(h(\cdot)\) is a smooth element-wise non-linear function; \(\lambda=\{w,u,b\}\) are free parameters; \(u\) represents a vector; \(w\) represents a weight vector; \(b\) represents a bias term, which is a parameter used to translate the input in the transformation;

[0035] Calculate its logarithmic Jacobian determinant in \(O(D)\) time: <o:p>< / o:p>

[0036] \(\psi(z)=h'(w T z + b)w

[0037]

[0038] where: \(\psi(z)\) represents the vector derived from the transformation function; \(h'(\cdot)\) is the derivative of the function \(h(\cdot)\); \(I\) is the identity matrix;

[0039] For the random variable \(z_0\) with the distribution \(q c (z_0|x)\), obtain the distribution of the random variable \(z_1 = f(z_0)\):

[0040]

[0041] Based on the properties of the Jacobian matrix of the invertible function, several simple mappings are combined through the chain rule and any arbitrarily complex distribution required is constructed sequentially; through the K transformation chain f K The random variable z0 with the distribution is sequentially transformed to obtain the target distribution as follows:

[0042] z K = f K (f K―1 …(f2(f1(z0))))

[0043]

[0044] where: z K represents the random variable obtained by sequentially transforming through the transformation chain f K .

[0045] Furthermore, in the fourth step, the target data set T = {(t m+1:n , x m+1:n )} is input into the latent path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution From which a random variable is sampled If f(z(t), t, θ) is Lipschitz continuous, then the deterministic Gaussian distribution is continuously transformed using neural ordinary differential equations to obtain:

[0046]

[0047] where: q(z(t i )) represents the probability density function of the latent variable z(t i ); f q (z(t i ), t i ) represents the rate function of the log probability density changing with time; f θ (z(t i ), t i ) represents the transformation function parameterized by the neural network; tr(·) represents the trace of the matrix; z(t i ) represents the variable of the latent variable z evolving at time t i ; t i represents the time point;

[0048] The target complex distribution that fits the true posterior is obtained by solving through the ODE solver and is expressed as:

[0049] log q N (z N|x) = log q(z(t i )) = ODESolve(log q(z(t0), f q , t0, t i )

[0050] where: q N (z N |x) represents the target complex distribution that fits the true posterior; ODESolve is a black-box differential equation solver for numerical integration methods; f q represents the rate function of the logarithmic probability density changing with time; t0 represents the initial time point.

[0051] Furthermore, in the fifth step, the loss function for improving the evidence lower bound is expressed as:

[0052]

[0053] where: p(x m+1:n |t 1:n , x 1:m ) represents the marginal likelihood of the target data point x m+1:n ; represents the expected value of the latent variable z; p(x i |z, t i ) represents the likelihood function of the target data point x m+1:n ; D kl (·) represents the Kullback-Leibler divergence that measures the difference between two probability distributions; q N (z N |x 1:n ) represents the target complex distribution that fits the true posterior; q c (z′|x 1:m ) represents the learnable multimodal prior distribution.

[0054] The beneficial effects of the present invention are as follows:

[0055] In modern manufacturing, the emergence of complex industrial equipment has further increased the cost of equipment condition assessment. Data-driven methods have overcome the difficulties of industrial equipment condition assessment to a certain extent. However, traditional data-driven methods are limited by the dependence on the simple Gaussian distribution assumption. When dealing with industrial scenarios with high-dimensional complex data characteristics, they cannot accurately characterize the true data distribution, resulting in the model being unable to fully exert its potential. To address the above problems, the present invention proposes an industrial equipment anomaly detection method based on a reliable inference attention neural process, aiming to accurately characterize the complex data distribution in real industrial scenarios. The method of the present invention is based on the attention neural process and abandons the Gaussian distribution assumption of the original attention neural process to eliminate inappropriate Gaussian priors and fuzzy Gaussian posteriors. The method of the present invention first introduces normalizing flow to achieve a learnable transformation of the prior distribution of the attention neural process, overcoming the hole problem caused by non-learnable priors. Secondly, the neural ordinary differential equation is used to smoothly transition the posterior distribution of the attention neural process, realizing the adaptive matching between the learnable prior distribution and the complex posterior distribution, and having the following technical effects:

[0056] (1) Solved the hole problem caused by the imperfect matching between non-learnable priors and complex posteriors for a long time. The method of the present invention has a solid mathematical foundation, can accurately characterize its prior, and perform reliable industrial equipment anomaly detection;

[0057] (2) The present invention studies the latest technologies of continuous non-linear bijections based on NODE to overcome the inaccurate approximation caused by the simple Gaussian posterior assumption of the attention neural process, improving the flexibility and accuracy of the model in approximating complex data distributions in real industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] To make the objectives, technical solutions, and beneficial effects of the present invention clearer, the following drawings are provided for illustration:

[0059] [[ID=*14]] Figure 1 It is the architecture diagram of the 3Ns model;

[0060] Figure 2 It is the schematic diagram of the learnable prior distribution based on NF;

[0061] Figure 3 It is the schematic diagram of the reliable posterior inference system based on NODE;

[0062] Figure 4 It is the loss curve diagram of different model frameworks;

[0063] Figure 5 It is the curve diagram of early anomaly detection performance;

[0064] Figure 6Comparison diagram of predicted joint motion signals (red) and true joint motion signals (blue) randomly generated for different models;

[0065] Figure 7 Diagram of the experimental device;

[0066] Figure 8 Mean absolute error (MAE) obtained by 3NNP through 6 repetitions of 10-fold cross-validation. Along the x-axis, each box represents a random partition, and the y-axis measures the range of MAE values for all 10 folds;

[0067] Figure 9 Performance of early anomaly detection for different models;

[0068] Figure 10 Comparison of predicted anomaly signals (left) with normal (right) and uncertainty quantification. Detailed implementation manners

[0069] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments cited are not intended to limit the present invention.

[0070] 1. Method

[0071] In this embodiment, a reliable estimation of the approximate posterior distribution of the Attentive Neural Processes (ANP) model and its assumed prior distribution is realized based on Neural Ordinary Differential Equations (NODE) and Normalizing Flows (NFs), thereby constructing an Attentive Neural Process model based on reliable inference to better realize the anomaly monitoring of key components of CNC machine tools.

[0072] 1.1. Method flow

[0073] This embodiment constructs a 3N model based on Neural Ordinary Differential Equations (NODE), Normalizing Flows (NFs) and Attentive Neural Processes (ANP) to realize the anomaly monitoring of key components during the operation of industrial equipment, and its structure is as Figure 1 shown. Specifically, the industrial equipment anomaly detection method based on reliable inference of Attentive Neural Processes includes the following steps.

[0074] Step 1: For the dataset of a given industrial equipment Define the joint distribution and construct a latent variable generation model based on the attention neural process.

[0075] Specifically, the dataset of the given industrial equipment is represented as:

[0076]

[0077] where: x i is the observed data point; represents the event space of all observed data points; Ω represents the sampling space; t i is the time; n is the number of data points;

[0078] The joint distribution is defined as:

[0079]

[0080] where: represents the joint distribution of the observed data point x i ; p represents the abstract probability distribution of the observed data point x i ; p(f) represents the marginal probability density function of the observed data point x i ; p(x i |f,t i ) represents the conditional probability density function of the observed data point x i under the condition of the given transformation f and time t i ; x i represents the observed data point; t i represents the set of all time points; f represents the transformation.

[0081] To represent random events using ANP approximate it with a neural network assuming that can be parameterized by a high-dimensional random vector z, and add some observation noise to replace the original X i = F(t i ), and then represent F(t) = g(t,z) as a certain fixed learnable function g, (i.e., the randomness in

[0082]

[0083] where: p(z|t i ,x i ) represents the given all observed time points t i and its corresponding observed data x iUnder the condition of, the posterior probability distribution of the latent variable z; p(z) is the prior probability distribution; Generate the observed data point x for the learnable function g after adding noise i Probability density; t i Denote the set of all time points, that is, each observed data x i The corresponding time t i .

[0084] Step two: Divide the data set Into the context data set C and the target data set T, and use amortized variational inference to learn the variational posterior of the latent variable, obtaining the evidence lower bound of the attention neural process.

[0085] p(z) is the prior probability distribution. However, in most industrial scenarios, we cannot obtain accurate prior knowledge. Therefore, p(z) is usually assumed to be a Gaussian distribution. Generate the observed data point x for the learnable function g after adding noise i Probability. But p(z|t 1:n ,x 1:n ) is intractable. We cannot optimize the neural network By maximizing the log-likelihood of . Therefore, we divide the data set Into the context data set C = {(x 1:m ,t 1:m )} and the target data set T = {(x m+1:n ,t m+1:n )}, and then use amortized variational inference to learn it, thereby obtaining the variational posterior q(z|t 1:m ,x 1:m ) of the latent variable z, that is, obtaining the evidence lower bound of the attention neural process, denoted as:

[0086]

[0087] For better understanding, perform simple algebraic operations on it to obtain a clearer evidence lower bound (EBLO):

[0088]

[0089] Where: p(x m+1:n |t 1:n ,x 1:m ) represents the conditional probability of predicting the target data set x m+1:n ,t m+1:n ) given the context data set {(x m+1:n}; Represents the true posterior distribution q(z|t 1:n ,x1:n )'s expectation; p(x i |z, t i ) represents the likelihood given the latent variable z and the time point t i ; D kl (·) represents the Kullback–Leibler divergence that measures the difference between two probability distributions; q(z|t 1:m , x 1:m ) represents the distribution that uses the variational posterior to replace the prior; q(z|t 1:n , x 1:n ) represents the true posterior distribution; C = {(x i , t i ), i ∈ [1, m]}, that is, C = {(x 1:m , t 1:m )} represents the context dataset; T = {(x i , t i ), i ∈ [1, n]}, that is, T = {(x m+1:n , t m+1:n )} represents the target dataset.

[0090] Step 3: Input the context dataset C into the deterministic path encoder of the latent variable generation model to obtain a prior distribution in the form of a simple Gaussian distribution; introduce normalizing flows and transform the prior distribution into a learnable multimodal prior distribution through reversible transformations.

[0091] However, since the distribution (z|t 1:m , x 1:m ) that uses the variational posterior to replace the prior and the true posterior distribution q(z|t 1:n , x 1:n ) both appear in the form of a multivariate standard Gaussian distribution in the ANP, however, in the actual industrial field, the true data distribution largely does not follow a multivariate Gaussian distribution. Therefore, this assumption cannot be well applied to the anomaly monitoring task of industrial equipment.

[0092] This embodiment proposes Model 3N. Using NODE, the original Gaussian distribution can be smoothly transformed into an arbitrarily complex distribution, thus better fitting the true posterior of the ANP. First, use the form of NFs to specify the assumed prior distribution required for variational inference. By inputting the context dataset C = {(t 1:m , x 1:m )} into the deterministic path encoder of the ANP model, encode the original signal to obtain a prior distribution q c (z|x) in the form of a simple Gaussian distribution, sample a random variable z0 from it, and then obtain an increasingly complex multimodal prior distribution q c(z′|x).

[0093]

[0094] where: q c (z′|x) represents a multimodal prior distribution; q c (z|x) is a prior distribution in the form of a simple Gaussian distribution; z is a random variable.

[0095] Step 4: Input the target dataset T into the latent path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution; use a neural ordinary differential equation to perform a continuous transformation on the deterministic Gaussian distribution, and solve it through an ODE solver to obtain the target complex distribution that fits the true posterior.

[0096] Specifically, input the target dataset T = {(t m+1:m , x m+1:n )} into the latent path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution Sample a random variable from it If f(z(t), t, θ) is Lipschitz continuous, then use a neural ordinary differential equation to perform a continuous transformation on the deterministic Gaussian distribution to obtain:

[0097]

[0098] where: q(z(t i )) represents the probability density function of the latent variable z(t i ); f q (z(t i ), t i ) represents the rate function of the logarithmic probability density changing with time; f θ (z(t i ), t i ) represents the transformation function parameterized by a neural network; tr(·) represents the trace of a matrix; z(t i ) represents the variable of the latent variable z evolving at time t i ; t i represents the time point.

[0099] Solve through an ODE solver to obtain the target complex distribution that fits the true posterior, expressed as:

[0100] log q N (z N |x) = log q(z(t i )) = ODESolve(log q(z(t0), f q , t0, t i )

[0101] where: q N (z N |x) represents the target complex distribution that fits the true posterior; ODESolve is a black-box differential equation solver for numerical integration methods; f q represents the rate function of the log probability density varying with time; t0 represents the initial time point.

[0102] Step 5: Construct an improved evidence lower bound that includes a multimodal prior term and the target complex distribution that fits the true posterior as the loss function, and perform model training through the reparameterization trick;.

[0103] The loss function of the improved evidence lower bound is expressed as:

[0104]

[0105] where: p(x m+1:n |t 1:n ,x 1:m ) represents the marginal likelihood of the target data point x m+1:n ; represents the expected value of the latent variable z; p(x i |z,t i ) represents the likelihood function of the target data point x m+1:n ; D kl (·) represents the Kullback-Leibler divergence that measures the difference between two probability distributions; q N (z N |x 1:n ) represents the target complex distribution that fits the true posterior; q c (z′|x 1:m ) represents the learnable multimodal prior distribution.

[0106] Since the 3Ns model is a probability model, the predictions output by the model are obtained by sampling a prediction distribution for the given data. To make the prediction distribution fit the target distribution as closely as possible, that is, to maximize the log likelihood log p(x m+1:n |t 1:n ,x 1:m ), however, since the original log likelihood function is difficult to handle, the reparameterization trick is used to estimate the lower bound of the log likelihood function to overcome this problem, and the learning of the model parameters is achieved by maximizing the evidence lower bound of the model.

[0107] Step 6: Use the trained model to calculate the conditional probability density of the new observed data, and achieve device anomaly detection through threshold judgment.

[0108] 1.2. Normalizing Flows (NFs)

[0109] To further explain the role of NF, the loss function is rewritten in the following form:

[0110]

[0111] The summation of the training data is explicitly emphasized, that is, the expected value of x with respect to the empirical distribution where δ(·) is the Dirac function. The new ELBO consists of two parts, namely the reconstruction error:

[0112]

[0113] and the regularization term between the encoder and the prior:

[0114]

[0115] Furthermore, by making some derivation transformations on the regularization term Ω, we can obtain:

[0116]

[0117] where is the cross-entropy between the posterior and the prior, is the conditional entropy between q(z N |x 1:n ) and the empirical distribution p(x). Since the optimization objective is to maximize the ELBO and there is a negative cross-entropy in the regularization term, the objective then becomes to minimize So the optimization result will be that the cross-entropy forces the posterior to match the prior. If the prior is not learnable (a fixed simple Gaussian distribution), the prior will be like an anchor point and the posterior will move like an amoeba to adapt to the prior. In practice, because decoding forces the encoder to reach a peak but finally it is impossible to perfectly match the fixed-shaped prior, we will get holes, that is, regions where the posterior assigns low probability and the prior assigns high probability in the latent space. This problem is particularly obvious in generative models because sampling from the prior holes may result in extremely low-quality samples. Therefore, we introduce NFs so that the model has a sufficiently flexible learnable prior, enabling the prior distribution and the posterior distribution to match each other, thus avoiding the occurrence of the hole problem.

[0118] Specifically, as Figure 2 shown, in this embodiment, the method of introducing a normalizing flow to convert the prior distribution into a learnable multimodal prior distribution through a reversible transformation is as follows.

[0119] Assume that we have an initial distribution q(z0), then the path traversed by the random variable z K = f K (z K―1 ) is called a flow, and the continuous distribution q KThe resulting path is called a normalized flow. The normalized flow can achieve any transformation of the distribution shape. Therefore, from the context dataset C = {(t 1:m ,x 1:m )} Randomly select a certain number of data points m, input them into the determined path of the attention neural process, encode the original signal to obtain a simple Gaussian distribution in the form of a prior distribution q c (z|x), sample a random variable z0 from it, introduce the normalization flow, and use the transformation group to perform shape transformation:

[0120] f(z)=z0+uh(w T z0+b)

[0121] Where: f(z) represents the new variable obtained by applying a set of transformations; z0 is a sampled random variable; h(·) is a smooth element-by-element nonlinear function; λ = {w,u,b} is a free parameter; u represents a vector or; w represents a weight vector; b represents the bias term, which is a parameter used to translate the input in the transformation.

[0122] Using this mapping, its logarithmic Jacobian determinant can be calculated in O(D) time:

[0123] ψ(z)=h′(w T z+b)w

[0124]

[0125] Where: ψ(z) represents the vector derived from the transformation function; h′(·) is the derivative of the function h(·); and I is the identity matrix.

[0126] For a distribution q c The random variable z0 of (z0|x) obtains the distribution of random variable z1=f(z0):

[0127]

[0128] Based on the properties of the Jacobian matrix of the reversible function, it is realized by the chain rule (inverse function theorem). Combining several simple mappings and sequentially constructing the required arbitrarily complex distribution; through the K transformation chain f K will have distribution The target distribution obtained by sequentially transforming the random variable z0 for:

[0129] z K =f K (f K―1 …(f2(f1(z0))))

[0130]

[0131] where: z K represents the random variable obtained by successive transformation through the transformation chain f K in sequence.

[0132] 1.3 Neural Ordinary Differential Equations (NODE)

[0133] For traditional deep learning models, it is found that the problem of gradient vanishing can be avoided through residual connection operations. This effective optimization measure can be expressed as follows:

[0134]

[0135] where: t ∈ {0…T}; represents that the dimension of the hidden layer is D; represents the t-th layer neural network with parameters θ t . These iterative updates can be regarded as the Euler discretization of continuous transformation. Since the powerful expressive ability of deep neural networks stems from the number of their hidden layers, when taking the limit case of an infinite number of layers, the residual network in the continuous case is obtained, which is the NODE. Rewrite the above formula as:

[0136] Δh t = f θ (h t )·Δt

[0137] where: Δt is the difference between layers. When the number of hidden layers t approaches infinity, the layer difference Δt will approach zero. Therefore, it can be rewritten in differential form as:

[0138] dh(t) = f θ (h(t), t)·dt

[0139] where t ∈ [0, T]. Integrating both sides of the above formula gives the output h T of the last layer:

[0140]

[0141] It can be seen from the above formula that when the number of hidden layers of the neural network approaches infinity, the way of transmitting information is not discretely propagated like that of a finite-layer network, but is expressed in the form of continuous integration. However, the computer cannot directly perform integral operations on the above formula to obtain its analytical solution h T , so we use numerical methods for ordinary differential equations to solve its numerical solution h T :

[0142]

[0143] Among them: ODESolve is a black-box differential equation solver for numerical integration methods. It discretizes the continuous information propagation process into a series of discrete time steps and simulates the information propagation by iteratively updating the system state. Therefore, ODESolve enables us to effectively process continuous dynamic systems on a computer. We only need to give the initial state h0, the neural network f θ (h(t), t), and the number of layers T. ODESolve can automatically propagate h0 forward along the continuous layers and output h T .

[0144] The introduction of NODE is mainly because when the attention neural process model learns and fits the data distribution, it often realizes it by assuming that the data distribution follows a simple distribution such as Gaussian, which greatly weakens the model's representation ability for data. Therefore, we do not pre-assume the specific form of the data distribution. By introducing NODE, we can freely change the shape of the distribution and gradually approximate (characterize) the complex distribution of high-dimensional data through the powerful and flexible data distribution fitting ability of the attention neural process to include the true posterior as a solution, thereby further enhancing the model's representation ability for data. Next, we will show how to use NODE to achieve the transformation of the distribution, as Figure 3 shown.

[0145] Input the target data set T = {(t m+1:n , x m+1:n )} into the latent layer path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution Sample a random variable from it If f(z(t), t, θ) is Lipschitz continuous, then use the neural ordinary differential equation to perform a continuous transformation on the deterministic Gaussian distribution to obtain:

[0146]

[0147] Among them: q(z(t i )) represents the probability density function of the latent variable z(t i ); f q (z(t i ), t i ) represents the rate function of the logarithmic probability density changing with time; f θ (z(t i ), t i ) represents the transformation function parameterized by the neural network; tr(·) represents the trace of the matrix; z(t i ) represents the variable of the latent variable z evolving at time t i ; t i represents the time point;

[0148] The target complex distribution that fits the true posterior is obtained by solving through an ODE solver, expressed as:

[0149] log q N (z N |x) = log q(z(t i )) = ODESolve(log q(z(t0), f q , t0, t i ).

[0150] Where: q N (z N |x) represents the target complex distribution that fits the true posterior; ODE represents the ODE solver; Solve(·) represents the Solve function; f q represents the rate function of the logarithmic probability density varying with time; t0 represents the initial time point.

[0151] To improve the generalization of the model, during training, we randomly select the number of data points in the context and target datasets, and the number of layers of the NODE also changes with the number of data points m in the training dataset. When there are more data points, a deeper network is required to achieve better fitting, which further saves computational resources.

[0152] 2. Experimental Verification

[0153] 3.1. Anomaly Detection for Industrial Robots

[0154] 3.1.1. Dataset

[0155] In this embodiment, all industrial robot data is collected from the welding robot control system at the production site. The welding robot control system contains joint current and rotational speed data obtained from the robot joints. For training and testing, we record the joint rotational speed and joint current of six joints of the robot from the robot controller at a frequency of 250 Hz. We select the rotational speed and current signals as input data. Each observation is a 12 - dimensional vector, including 6 joint motor current signals and 6 joint motor rotational speed signals, and each signal contains 650 sampling points. The training dataset consists of 8400 consecutive normal - period signals, and the model verification dataset consists of 1918 consecutive normal - period signals and 882 abnormal - period signals. Since our goal is anomaly detection, we only use the data generated by the robot during normal operation to train the model to ensure that the model only learns features from normal data. Therefore, the training dataset can be expressed as D = {X|X ∈ R 8400*650*12}

[0156] 3.1.2. Results and Discussion

[0157] The experiment was conducted on a computing workstation equipped with an AMD Ryzen Threadripper PRO 5965WX CPU, 256GB of RAM, and two NVIDIA RTX A6000 GPUs. Specifically, the prediction time was approximately 1 second, which is negligible compared to the 10-minute manufacturing time of a vehicle in this automotive production line.

[0158] To verify the effectiveness of the model proposed modules, we used the method of controlling a single variable to conduct ablation experiments on the two proposed modules in sequence. The results are shown in Table 1. First, we verified the effectiveness of the proposed method of using neural ordinary differential equations to smoothly transition the posterior distribution of neural processes. Compared with the original attention neural process, after the smooth transition of the posterior distribution by NODE in our model, all performance indicators have been significantly improved. In particular, the AUC value, which demonstrates the comprehensive performance of the model, has increased by 12%. This indicates that the change in the complex posterior distribution has further improved the model's ability to fit the data distribution in industrial scenarios. However, we can see that the values of its precision and accuracy have only increased slightly. This is because the non-learnable simple Gaussian prior distribution forms "holes" when fitting the complex real distribution, resulting in extremely low sample sampling quality.

[0159] Secondly, we further verified the contribution of the proposed method of learnable prior distribution transformation based on normalizing flow to the model. As shown in Table 1, compared with the original attention neural process model, the model that only uses normalizing flow for distribution transformation has significantly improved in all performance aspects. This is because we use normalizing flow to achieve the transformation of complex prior distributions, providing the model with the flexibility to adapt to complex data distributions, thereby improving the model's ability to fit the complex distributions of highly heterogeneous industrial data.

[0160] Table 1: Performance of Model Components

[0161] Ablation experiment p R F1 Accuracy AUC ANP 0.8365 0.9259 0.8864 0.8022 0.75 3NNP w / o NODE 0.8367 0.9845 0.9046 0.8268 0.87 3NNP w / o flow 0.8670 0.9930 0.9257 0.8671 0.77 3NNP 0.9148 1.0000 0.9555 0.9222 0.97

[0162] To strictly verify the superiority and reliability of the proposed model, we carefully selected a series of generative models for comparison as baseline models. Our work focuses on how to improve the model's ability to depict the real distribution. Therefore, the baseline models we selected especially considered neural processes and their variants, aiming to demonstrate the effectiveness of the proposed method. First, we deeply studied the convergence ability of the model, from Figure 4It can be seen that compared with ANP and NP, the model framework we proposed not only significantly improves the model performance but also has a convergence speed comparable to that of NP and is faster than ANP. This indicates that our model framework has a stronger fitting ability for the true complex distribution of data in industrial scenarios and a faster learning speed.

[0163] Secondly, on this basis, we conducted experiments on other baseline models. The results are shown in Table 2. The performance of the model framework we proposed has been comprehensively improved compared with ANP. The accuracy has increased by 12%. This is because our proposed model performs distribution transformations on the prior distribution and posterior distribution in its simple Gaussian form, respectively, thus improving its ability to depict complex distributions. However, the indicators of NP are not high, because NP itself has the deficiency of underfitting, and coupled with the simple Gaussian distribution assumption, it further weakens the ability of the model to characterize the true distribution of data in industrial scenarios. Finally, we also conducted comparative experiments on the remaining generative models. It can be seen that the performance of VAE and its variant CVAE is generally not high. This is because in order to approximate the scenario of few negative samples in the real industrial scenario during testing, we used small-sample experiments for all models. It can be seen that although VAE is a mature and powerful generative model, it cannot handle the equipment anomaly detection task in industrial scenarios well. Finally, we conducted experiments on GAN. It can be seen that the performance of the GAN model is extremely unbalanced, which is caused by the fact that real industrial data is not only small-sample but also high-dimensional.

[0164] Table 2: Performance of Different Methods

[0165]

[0166]

[0167] To verify the ability of the model framework we proposed in realizing early anomaly detection of industrial equipment, we conducted early anomaly detection experiments. We tested the ability of the model to detect anomalies when only 10%, 20%, 40%, 60%, 80%, and 95% of the observed data were given respectively. From Figure 5 it can be seen that when our proposed model has only 10% of the observed data points, the F1 value for equipment anomaly detection reaches 89.09%, and the recall rate reaches 98.99%. Moreover, as the number of observed data points increases, the performance indicators of the model are further improved. It can be seen that the model framework we proposed has a certain ability for early anomaly detection of equipment and can realize the early warning function for equipment anomalies.

[0168] As a generative model with powerful distribution modeling capabilities, the generative ability of the neural process strongly demonstrates the model's ability to capture the underlying data distribution of robotic joint motor signals and its profound understanding of robot faults. However, limited by its simple Gaussian distribution assumption and underfitting, the neural process cannot learn the complex distribution of robot data in real industrial scenarios. The attention neural process well overcomes the underfitting problem by adding an attention mechanism, but it still cannot depict the complex distribution details of the data. By leveraging the smooth transition of the posterior distribution using neural ordinary differential equations and realizing the learnable transformation of the prior distribution using normalizing flows, our model has well learned the detailed features of the complex data distribution in real industrial scenarios, further deepening the model's understanding of robot faults. Figure 6 The predicted signals (red) and true signals (blue) generated by each model are shown. It can be seen that the signals generated by our model capture all the variation trends and slight perturbations of the true robot signals, which verifies that the model has perfectly learned the true complex distribution of the data using distribution transformation.

[0169] 3.2. Detection of Abnormalities in the Ball Screw of CNC Machine Tools

[0170] 3.2.1. Datasets

[0171] To further verify the model's ability to detect abnormalities in different industrial equipment, we collected the status monitoring signal data in the CNC system of a Siemens 828D model CNC machine tool using the NiceNet intelligent gateway G300 at a frequency of 10 Hz. The experimental settings are as Figure 7 shown. Secondly, to better fit the situation where the amount of negative sample data is scarce in the small-sample abnormality detection task in real industrial scenarios, we collected 100 cycles of normal signals of the CNC machine tool under loads of 0 kg, 10 kg, 20 kg, 40 kg, and 50 kg, and 60 cycles of abnormal signals under three types of failed ball screws: contact fatigue failure, wear failure, and memory fatigue fracture. Each cycle contains 70 sampling points. Since our task is abnormality detection, during the training phase, we only use 500 cycles of normal signal data to train the model. Therefore, the training dataset can be expressed as M = {X|X ∈ R 500*70*3}. During the testing phase, we will randomly select 180 cycles from the normal signals and 20 cycles from the abnormal signals to verify the abnormality detection performance of the model in the small-sample case. Therefore, the test dataset can be expressed as N = {X|X ∈ R 200*70*3}.

[0172] 3.2.2. Results and Discussions

[0173] To verify the generalization ability and stability of the model in the fault inference of the ball screw of CNC machine tools, we repeated the 10-fold cross-validation 6 times on the dataset. The mean absolute error (MAE) results of each test are as Figure 8 shown. First, it can be seen that the median lines of all boxes are located near 0.04, indicating that the errors of our model show stable performance in most ball screw fault inferences. Second, the boxes of the 10-fold cross-validation repeated 6 times are not very different, indicating that the model has relatively good generalization ability and consistency in different dataset partitions and is not affected by the dataset distribution. Finally, it can be seen that the average MAE (red dashed line) is very close to the median line (green dashed line), which means that our model has no systematic bias. These results ensure that the model can reliably predict and generalize to new data in a wide range of manufacturing scenarios.

[0174] To verify the overall performance of the model in the small-sample anomaly detection task under class imbalance, we further compared metrics such as RMSE and AUC with other models. As shown in Table III, first, our model has the lowest RMSE, which indicates that 3NNP captures the key features in the data more fully than other models and learns well the complex underlying distribution from the input data. Second, the precision and recall rates of 3NNP exceed those of other models by more than 15%, far higher than those of other comparison models, indicating that our model solves well the common class imbalance problem in real industrial scenarios. Finally, the AUC value of 3NNP has increased by 6% compared with the original model ANP, reaching 96%, which shows that the model can give a higher prediction probability to the true positive class and a lower probability to the true negative class, further proving that the model has strong generalization ability.

[0175] Table 3: Performance of different methods

[0176] Model RMSE P AUC F1 Accuracy 3NNP 0.0771 0.96 0.96 0.9373 0.9124 ANP 0.2613 0.7496 0.90 0.8562 0.7601 NP 0.3952 0.7149 0.64 0.8310 0.7112 VAE 0.3469 0.6690 0.65 0.7289 0.6683 CVAE 0.2183 0.7842 0.70 0.7826 0.7657 GAN 0.6993 0.6284 0.60 0.6763 0.4997

[0177] To further verify the contribution of the proposed module in the early anomaly detection of equipment, we selected the models ANP and NP to compare the ability of the models to detect early anomalies, and selected two metrics F1 and AUC that can comprehensively characterize the model performance to characterize the ability of the models to detect early anomalies. From Figure 9 it can be seen that when only 10% of the observed data points are given, the F1 and AUC values of the proposed model far exceed those of NP and ANP, indicating that the proposed module has a very significant effect in improving the early anomaly detection ability of the model.

[0178] Finally, we conducted a visualization experiment on the equipment anomaly detection ability of the proposed model and its ability to quantify the uncertainty of anomaly prediction. From Figure 10As can be seen on the right side, when the model is given normal observation data points, since it has fully learned the complex latent distribution of the normal data, the model can well predict the complete signal, and the uncertainty of the predicted data points is very small, indicating that at this point the model has a high degree of confidence in identifying it as a normal data point. When an abnormal observation data point appears (within the red frame), since the model has not learned its latent distribution, it cannot well predict other data points. Whether the device is abnormal is determined by judging whether the probability density value under the normal signal distribution exceeds the threshold, and the uncertainty of the model's prediction of it is very large, which means that our model has only a very small chance of predicting this abnormal data point as a normal data point.

[0179] 3. Conclusion

[0180] Aiming at the problem that the existing data-driven models cannot effectively characterize the high-dimensional data features of modern complex industrial equipment due to the simple Gaussian distribution assumption, this paper proposes a new probability model 3NNP for learning and inferring the true distribution of high-dimensional data of complex industrial equipment. First, aiming at the sampling hole problem caused by the unlearnable prior distribution, we use normalizing flows to enable the model to obtain a flexible and learnable prior distribution; second, we use neural ordinary differential equations to achieve a smooth transition of the posterior distribution, so that the prior distribution and the posterior distribution can be perfectly matched; finally, we verify the effectiveness and flexibility of the proposed method for abnormal detection of industrial equipment in real industrial scenarios through industrial robot abnormal detection (status assessment) and CNC machine tool ball screw abnormal detection (status assessment) experiments.

[0181] The above-described embodiments are only preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. An industrial equipment anomaly detection method based on a reliable inference attention neural process, characterized in that: It includes the following steps: Step 1: For the dataset of a given industrial device Define the joint distribution and construct a latent variable generation model based on the attention neural process; Step 2: Divide the dataset into a context dataset C and a target dataset T, and use amortized variational inference to learn the variational posterior of the latent variable, obtaining the evidence lower bound of the attention neural process; Step 3: Input the context dataset C into the deterministic path encoder of the latent variable generation model to obtain a prior distribution in the form of a simple Gaussian distribution; introduce normalizing flow, and convert the prior distribution into a learnable multimodal prior distribution through a reversible transformation; Step 4: Input the target dataset T into the latent path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution; use a neural ordinary differential equation to perform a continuous transformation on the deterministic Gaussian distribution, and solve it through an ODE solver to obtain a target complex distribution that fits the true posterior; Step 5: Construct an improved evidence lower bound containing the multimodal prior term and the target complex distribution that fits the true posterior as the loss function, and perform model training through the reparameterization trick; Step 6: Use the trained model to calculate the conditional probability density of the new observed data, and achieve device anomaly detection through threshold judgment.

2. The industrial equipment anomaly detection method based on a reliable inference attention neural process according to claim 1, wherein: In the first step, the dataset of the given industrial equipment is expressed as: where: x i is the observed data point; represents the event space of all observed data points; Ω represents the sampling space; t i is the time; n is the number of data points; The joint distribution is defined as: Wherein: represents the joint distribution of the observed data points x i ; p represents the abstract probability distribution of the observed data points x i ; p(f) represents the marginal probability density function of the observed data points x i ; p(x i |f,t i ) represents the conditional probability density function of the observed data points x i under the condition of the given transformation f and time t i ; x i represents the observed data points; t i represents the set of all time points; f represents the transformation; Representing Random Events Using Attention Neural Processes Suppose can be parameterized by a high-dimensional random vector z and add observation noise to replace X i = F(t i ); Represent F(t) = g(t, z) as a certain fixed learnable function g to obtain a latent variable generation model: where: p(z|t i ,x i ) represents the posterior probability distribution of the latent variable z given all the observed time points t i and their corresponding observed data x i ; p(z) is the prior probability distribution; is the probability density of the learnable function g generating the observed data point x i after adding noise; t i represents the set of all time points, i.e., the time t i corresponding to each observed data x i .

3. The industrial equipment anomaly detection method based on a reliable inference attention neural process according to claim 1, characterized in that: In step 2, the evidence lower bound of the attention neural process is expressed as: where: p(x m+1:n |t 1:n ,x 1:m ) represents the conditional probability of predicting the target dataset x m+1:n ,t m+1:n ) given the context dataset {(x m+1:n ; represents the expectation of the true posterior distribution q(z|t 1:n ,x 1:n ); p(x i |z,t i ) represents the likelihood given the latent variable z and the time point t i ; D kl (·) represents the Kullback-Leibler divergence that measures the difference between two probability distributions; q(z|t 1:m ,x 1:m ) represents the distribution that uses the variational posterior to replace the prior; q(z|t 1:n ,x 1:n [[ID=3y]]) represents the true posterior distribution; C = {(x i ,t i ), i ∈ [1, m]}, that is, C = {(x 1:m ,t 1:m )} represents the context dataset; T = {(x i ,t i ), i ∈ [1, n]}, that is, T = {(x m+1:n ,t m+1:n )} represents the target dataset.

4. The industrial equipment anomaly detection method based on a reliable inference attention neural process according to claim 1, wherein: In step 3, the obtained multimodal prior distribution is expressed as: where: q c (z′|x) represents a multimodal prior distribution; q c (z|x) is a prior distribution in the form of a simple Gaussian distribution; z is a random variable.

5. The industrial equipment anomaly detection method based on a reliable inference attention neural process according to claim 4, characterized in that: The method of introducing normalizing flow and converting the prior distribution into a learnable multimodal prior distribution through a reversible transformation is: Input the context dataset C = {(x 1:m , t 1:m )} into the deterministic path encoder of the latent variable generation model to obtain the prior distribution q c (z|x) in the form of a simple Gaussian distribution; sample a random variable z0 from it, introduce a normalizing flow, and perform shape transformation using a transformation group: f(z) = z0 + uh(w T z0 + b) Where: f(z) represents the new variable obtained by applying a set of transformations; z0 is a randomly sampled variable; h(·) is a smooth element-wise non-linear function; λ = {w, u, b} are free parameters; u represents a vector or; w represents a weight vector; b represents a bias term, which is a parameter used to translate the input in the transformation; Calculate its logarithmic Jacobian determinant within O(D) time: ψ(z) = h′(w T z + b)w Where: ψ(z) represents the vector derived from the transformation function; h′(·) is the derivative of the function h(·); I is the identity matrix; For a random variable z0 with distribution q c (z0|x), obtain the distribution of the random variable z1 = f(z0): Based on the properties of the Jacobian matrix of invertible functions, several simple mappings are combined through the chain rule and any arbitrarily complex distribution required is constructed sequentially; through the K transformation chain f K The random variable z0 with the distribution is sequentially transformed to obtain the target distribution as follows: z K = f K (f K―1 …(f2(f1(z0)))) where: z K represents the random variable obtained by successive transformations through the transformation chain f K in sequence.

6. The industrial equipment anomaly detection method based on a reliable inference attention neural process according to claim 1, wherein: In the fourth step, the target data set \(T =\{(t m+1:n ,x m+1:n )\}\) is input into the latent path encoder of the latent variable generation model to obtain a deterministic Gaussian distribution From which a random variable is sampled If \(f(z(t),t,\theta)\) is Lipschitz continuous, then the neural ordinary differential equation is used to perform a continuous transformation on the deterministic Gaussian distribution to obtain: where: q(z(t i )) represents the probability density function of the latent variable z(t i ); f q (z(t i ),t i ) represents the rate function of the log probability density varying with time; f θ (z(t i ),t i ) represents the transformation function parameterized by a neural network; tr(·) represents the trace of a matrix; z(t i ) represents the variable of the latent variable z evolving at time t i ; t i represents a time point; Solve through an ODE solver to obtain a target complex distribution that fits the true posterior, expressed as: log q N (z N |x) = log q(z(t i )) = ODESolve(log q(z(t0), f q , t0, t i ) where: q N (z N |x) represents the target complex distribution that fits the true posterior; ODESolve is a black-box differential equation solver for numerical integration methods; f q represents the rate function of the log probability density changing over time; t0 represents the initial time point.

7. The industrial equipment anomaly detection method based on a reliable inference attention neural process according to claim 1, characterized in that: In step 5, the loss function of the improved evidence lower bound is expressed as: where: p(x m+1:n |t 1:n ,x 1:m ) represents the marginal likelihood of the target data point x m+1:n ; represents the expected value of the latent variable z; p(x i |z,t i ) represents the likelihood function of the target data point x m+1:n ; D kl (·) represents the Kullback–Leibler divergence that measures the difference between two probability distributions; q N (z N |x 1:n ) represents the target complex distribution that fits the true posterior; q c (z′|x 1:m ) represents the learnable multimodal prior distribution.

Images