A method and apparatus for robust fault diagnosis of missing values ​​based on spatiotemporal fusion

Abstract

This invention discloses a method and apparatus for fault diagnosis based on spatiotemporal fusion, which is robust to missing values. The method includes: a fault diagnosis model based on spatiotemporal fusion, utilizing temporal correlations in the data and spatial correlations between sensors, and using a deep autoencoder to extract features, obtaining a feature extraction model that minimizes signal reconstruction error; acquiring the spatiotemporal correlation characteristics of the time series, and using a long short-term neural network cell structure with memory for temporal correlations, enabling the feature extraction model to perform fusion analysis of spatiotemporal correlations; and detecting and analyzing faults based on the fused feature extraction model. The apparatus includes a processor and a memory. This invention can simultaneously extract the widespread temporal and spatial correlations in sensor network time series data, uncover more robust key data representations, and classify these representations. This invention can address the impact of missing value problems commonly found in sensor networks on the stability of fault diagnosis systems.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis, and in particular to a fault diagnosis method and apparatus based on spatiotemporal fusion that is robust to missing values. Background Technology

[0002] The development of next-generation communication technologies and the Industrial Internet of Things (IIoT) has led to the widespread application of various types of sensors in industrial production, resulting in an explosive growth in sensor data and laying the foundation for large-scale intelligent manufacturing. Fault diagnosis technology, by analyzing the correlations between sensor data and extracting key features, can accurately monitor the operational status of various production departments, detect operational anomalies in real time, and make effective judgments. This improves operational efficiency while reducing reliance on expert experience and saving labor costs, which is of great significance for intelligent and automated production. However, due to the complexity of actual production conditions and the high requirements for sensor reliability and long-term operating capabilities, sensor failures frequently occur. Furthermore, the large number of terminals can cause network congestion, leading to data transmission packet loss and data storage failures. Consequently, the collected data often contains a large number of missing values ​​of various types. The presence of these missing values ​​interferes with data analysis, greatly hinders the stability of real-time diagnostic systems, and significantly reduces the usability and value of fault diagnosis technology.

[0003] Current fault diagnosis methods either rely on mechanistic models for modeling and analysis, knowledge-based expert systems, or data-driven feature analysis. With the widespread use of sensors and the development of communication technologies, data acquisition, transmission, and storage have become easier, and feature extraction has become simpler and more convenient thanks to the development of deep learning. Therefore, data-driven approaches have received wider attention. For example, data-driven diagnostic methods such as PCA (Principal Component Analysis), KPCA (Kernel Principal Component Analysis), ICA (Independent Component Analysis), and KNN (k-Nearest Neighbors) rely on multivariate statistical analysis to construct relevant statistics and set control limits to detect and diagnose faults. Meanwhile, methods such as SVM (Support Vector Machine), ANN (Artificial Neural Network), and Bayesian networks, due to their data processing and classification capabilities, can effectively identify abnormal behavior and also play a role in fault detection. In addition, clustering methods, nearest neighbor methods, and spectral analysis techniques are also widely used in the field of fault detection.

[0004] The main challenges currently facing methods for imputing missing values ​​in sensor time-series data are:

[0005] On the one hand, industrial production processes are complex, data is abundant, and operating states change frequently, making it difficult to establish accurate predictive models. On the other hand, fault types are diverse, and new faults arise during machine operation. In some extreme cases, equipment failures can have devastating consequences, making the construction of expert systems requiring extensive diagnostic experience extremely difficult and costly. Data-driven methods have significant advantages in the current information age, but due to limitations imposed by non-fault factors in real-world conditions, the collected data often contains a large number of missing values. These missing values ​​severely affect the reliability and accuracy of fault diagnosis methods, leading to frequent misjudgments. Meanwhile, the analysis and diagnosis of fault data is costly, and in some cases, equipment failures can have devastating consequences, making it difficult to obtain usable tag data for machines. Furthermore, most existing methods only consider data relationships on a single dimension, failing to fully explore the characteristics of sensor data itself and the relationships between these characteristics, leaving room for further improvement in the accuracy and stability of fault diagnosis. Summary of the Invention

[0006] This invention provides a spatiotemporal fusion-based robust fault diagnosis method and apparatus for missing values. Based on a detailed analysis of the correlations in sensor time-series data, and combined with deep learning methods, this invention proposes a spatiotemporal fusion-based robust fault diagnosis model for missing values, MRFD. The MRFD model is data-driven and includes two modules: feature extraction and classification. It can simultaneously extract the widespread temporal and spatial correlations in sensor network time-series data, uncover more robust key data representations, and classify these representations for fault detection and analysis. This invention can address the impact of missing values, a common problem in sensor networks, on the stability of fault diagnosis systems. Details are described below:

[0007] A fault diagnosis method based on spatiotemporal fusion with robust missing values, the method comprising:

[0008] The fault diagnosis model based on spatiotemporal fusion utilizes the temporal correlation in the data and the spatial correlation information between sensors, and uses a deep autoencoder to extract features, thereby obtaining a feature extraction model that minimizes signal reconstruction error.

[0009] The spatiotemporal correlation characteristics of time series are obtained, and a long-short-term neural network cell structure with memory of time series correlation is used, so that the feature extraction model has the ability to perform fusion analysis of spatiotemporal correlation.

[0010] Fault detection and analysis are performed based on the fused feature extraction model.

[0011] The fault diagnosis model is as follows:

[0012] The feature extraction module is a DAE structure. The DAE encoding and decoding layers are LSTM sequences connected by LSTM neurons, and the intermediate layer is a feature vector containing core feature information after dimensionality reduction. The feature extraction module is trained by minimizing the error between the original data and the reconstructed data in order to extract the most robust representation in the data to be diagnosed.

[0013] The classification module is a DBN structure, consisting of four RBM layers and one BP layer. The RBM layers map the feature vectors to different feature spaces. Only the BP layer uses label information to train the fault classifier in a supervised manner, and the error is propagated down to fine-tune the network.

[0014] Furthermore, the loss function of the feature extraction model is:

[0015]

[0016] Where r(x)=σ(Wx+b) is the reconstruction of the input signal, σ is a nonlinear function, b is the neuron bias, x is the input signal, W is the neuron weight, and λ>0 is the regularization parameter that controls the smoothness Φ(x);

[0017]

[0018] Where r(x) j-2 ), r(x j+2 ), r(x j-1 ), r(x j+1 ) is the predicted value obtained after DAE reconstruction of adjacent timestamp inputs.

[0019] The backpropagation error term of the i-th neuron in the output layer of the feature extraction model is:

[0020]

[0021] Among them, W i l This represents the weight of the i-th neuron in the output layer.

[0022] Wherein, the output of the i-th node of the output layer of the DAE is o i The expected output is d i The correction error is then:

[0023] δ i =o i (1-o i )(d i -o i )

[0024] The correction error for the l-th hidden layer is calculated as follows:

[0025]

[0026] A fault diagnosis device based on spatiotemporal fusion for robust missing values, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform any of the steps of the method described above.

[0027] A computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform any of the steps of the method described above.

[0028] The beneficial effects of the technical solution provided by this invention are:

[0029] 1. The fault diagnosis model MRFD based on spatiotemporal fusion can simultaneously integrate and utilize two types of information: temporal correlation in the data and spatial correlation between sensors, to obtain a feature extraction model that minimizes signal reconstruction error. Therefore, the extracted representation is more robust to missing values ​​of the temporal signal and can stably detect and diagnose faults even when there are many failed sensors.

[0030] 2. For sensor data with different missing modes coexisting, the autoencoder structure of the feature extraction model does not need to model each missing mode separately, but acquires the core features of the data within a single frame. After one training, it can compensate for the impact of different types of missing values ​​on the accuracy of fault diagnosis.

[0031] 3. The MRFD model can make full use of unlabeled data for training. It can achieve good diagnostic accuracy by using only a small number of labels for supervised fine-tuning in the DBN (Deep Belief Network) classification layer, which reduces the dependence on labeled data and overcomes the problem of high difficulty in obtaining labels and scarcity of available labeled data in the field of fault diagnosis to a certain extent. Attached Figure Description

[0032] Figure 1 Here is the network structure diagram of MRFD;

[0033] Figure 2 The flowchart shows the training algorithm for the feature extraction module.

[0034] Figure 3 This is a schematic diagram of a weight-sharing strategy;

[0035] Figure 4 This is a schematic diagram of the DBN network structure;

[0036] Figure 5 This is another schematic diagram of the DBN network structure. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0038] Sensor data contains a variety of missing types, and different types of missing values ​​may coexist in large areas. MRFD's feature extraction module uses DAE (Deep Autoencoder) for feature extraction. DAE can automatically compress and reconstruct data. After training on massive amounts of data, the model possesses the ability to remove redundant and irrelevant information and extract the most important features. These features contain key information that can recover normal signals, thus eliminating the need for separate modeling for specific missing types. Instead, it can process data containing various types of missing values ​​simultaneously, enhancing the practicality of fault diagnosis. To obtain the spatiotemporal correlation characteristics of time series, an LSTM (Long Short-Term Neural Network) cell structure with memory for time series correlations is used on top of DAE, enabling the feature extraction model to perform fusion analysis of spatiotemporal correlations. DAE can learn unsupervised and does not require data labels during training. Simultaneously, during feature extraction, a smoothing regularization factor is used to introduce adjacent local temporal information, allowing it to simultaneously perceive global and local temporal correlations. Furthermore, by employing a weight-sharing strategy to train the feature extraction model, the smoothing regularization term shares the same weights across different time points, effectively reducing training complexity while maintaining the model's efficiency. The features obtained after dimensionality reduction are more robust, and training a deep belief network to extract and classify these features at a deeper level effectively distinguishes different anomaly types, minimizing the impact of missing values ​​on fault diagnosis applications.

[0039] The network structure of the fault diagnosis model MRFD proposed in this embodiment of the invention is as follows: Figure 1 As shown, it consists of two modules: feature extraction and classification. The feature extraction module is a DAE structure, where the DAE encoding and decoding layers are LSTM sequences formed by connecting LSTM neurons, and the intermediate layers are dimensionality-reduced feature vectors containing core feature information. The feature extraction model is trained by minimizing the error between the original data and the reconstructed data to extract the most robust representation from the data to be diagnosed. The classification module is a DBN structure, which is a probabilistic generative model consisting of four RBM (Restricted Boltzmann Machine) layers and one BP (Backpropagation) layer. The RBM layers map the feature vectors to different feature spaces to mine deeper feature information. Layer-by-layer training of the RBM does not require label information; only the BP layer uses label information to train the fault classifier in a supervised manner, and the error is propagated down to fine-tune the network.

[0040] The model training algorithm process is as follows:

[0041] 1) Raw data processing: The input raw data is successively divided into time series segments of length T;

[0042] The original data, after being sliced, yields the input data {x}. 1 x 2 x 3 , ...x T , where represents the multidimensional historical data from time 0 to time T. j ∈R M Let represent the sensor data at time j. It is an M-dimensional vector, and R is a real vector space.

[0043] 2) Initialize the feature extraction model;

[0044] 3) The DAE coding layer extracts the features of the input time segment and retains them in the intermediate layer. The decoding layer reconstructs the signal to obtain the reconstructed values ​​of the current and adjacent time steps. The reconstruction error, regularization term and backpropagation error are calculated using the input data and the reconstructed values.

[0045] 4) Update the neuron weights, input the data for the next time step and continue iterating until the feature extraction model is trained. Then remove the decoding part and retain the encoding part and feature vector.

[0046] 5) Initialize the classification model;

[0047] 6) The DAE encoding layer, intermediate layer and classification model are concatenated. The feature vector extracted by the feature extraction model is used as the input of the classification model. After training an RBM layer, the classification model uses the activation value of its hidden layer as the input of the next layer. Unsupervised pre-training is performed layer by layer, and the diagnostic results are output.

[0048] 7) The BP layer uses the diagnostic results and data labels to calculate the objective function and correct the errors of the output layer and hidden layer. It updates the weights layer by layer from top to bottom and performs supervised fine-tuning of the classification model.

[0049] While LSTM can capture global temporal correlations between sensors at different times, in actual system operation, situations not recorded in historical data may arise. Moreover, such low-probability faults often carry significant risks. Therefore, relying solely on global correlations from historical data may be insufficient to determine the existence of a current fault. In time series data, adjacent timestamps often exhibit a degree of similarity.

[0050] Therefore, in this embodiment of the invention, a smoothing regularization is added to the feature extraction model, such as... Figure 2As shown, this regularization term compensates for signal discontinuity caused by missing values ​​by penalizing the difference between predicted values ​​at adjacent timestamps. It uses time information that is as close as possible to the sensor signal value at adjacent times to perform feature extraction for the next classification step. The loss function of the feature extraction model is as follows:

[0051]

[0052] Where r(x) = σ(Wx + b) is the reconstruction of the input signal, σ is a nonlinear function, b is the neuron bias, x is the input signal, W is the neuron weight, and λ > 0 is the regularization parameter that controls the smoothness Φ(x).

[0053]

[0054] In this embodiment of the invention, the smoothing window size is set to 5, r(x) j-2 ), r(x j+2 ), r(x j-1 ), r(x j+1 ) is the predicted value obtained after DAE reconstruction of adjacent timestamp inputs.

[0055] The model training process in the feature extraction stage is as follows: Figure 3 As shown, when performing forward computation on data from adjacent time points, a forward network with the same weights is used to reduce computational complexity, resulting in predicted values ​​for adjacent time points. The backpropagation error term of the i-th neuron in the output layer of the feature extraction model is:

[0056]

[0057] Among them, W i l This represents the weight of the i-th neuron in the output layer. The last term is the weight of r(x) in W. i l The partial derivative at point L(r(x), x) is the loss function of the feature extraction model, and λ is the regularization parameter.

[0058] The DBN in the fault classification and diagnosis stage is an energy-based probabilistic generative model obtained by stacking multiple Restricted Boltzmann Machines (RBMs). By establishing a joint distribution of sensor data and fault labels, it possesses excellent feature extraction and fault classification capabilities, making it suitable for data-driven fault diagnosis needs. For an RBM layer, its visible layer v = {υ1, υ2, υ3, ..., υ...} m} and hidden layers h = (h1, h2, h3, ..., h n There are connections between layers, but no connections within layers. The energy function E of the joint state (v, h) is... θ (v, h) is:

[0059]

[0060] Where M and N are the number of visible and hidden layer units, respectively, and θ = {a, b, W} are the parameters of the RBM to be learned, where W nm Let a be the connection weight between the m-th visible layer unit and the n-th hidden layer unit. m and b n These represent the biases of visible and hidden cells, respectively. m Let h be the state vector of the m-th visible layer unit. n Let be the state vector of the nth hidden layer unit.

[0061] From the energy function, the joint probability density of (v, h) is:

[0062]

[0063] Where Z is the normalization constant:

[0064] Z = ∑ v ∑ h e -E(v，h|θ) (6)

[0065] Since the units in the visible layer and the hidden layer are independent of each other, P(v, h|θ) can be obtained as follows:

[0066]

[0067] The use of a five-layer DBN in the classification phase, such as Figure 4 and Figure 5 As shown, the first visible layer and the subsequent first hidden layer constitute an RBM layer. The result of its training is input into the second hidden layer, forming the second RBM. Unsupervised training is then performed layer by layer from bottom to top, enabling deeper feature extraction from the feature variables obtained by the feature extraction module. The classification model can be trained by maximizing the likelihood function, i.e.:

[0068] θ * =arg max θ ∑ m lnP(v|θ) (8)

[0069] Where, θ * The classification model parameters are used to maximize the likelihood function.

[0070] The parameters are iterated as follows:

[0071]

[0072] Where ε is the iteration step size.

[0073] The output of the last RBM layer of the DBN is used as the input to the BP layer of the supervised neural network, and the entire DBN is fine-tuned from top to bottom using label information. Let y be the true class of the fault data, o = h(w, b) be the output of the DBN, and the objective function be:

[0074] min w，b ||yh(w, b)|| 2 (10)

[0075] For the DBN output layer, assuming the output of the i-th node is o i The expected output is d i Then the correction error for that node is:

[0076] δ i =o i (1-o i )(d i -o i (11)

[0077] The correction error for the l-th hidden layer is calculated as follows:

[0078]

[0079] Let ∈ be the iteration step size, and the weight update is calculated as follows:

[0080]

[0081]

[0082] After the MRFD model is trained in modules according to the above steps, it can be used for fault identification and diagnosis using sensor time-series data in the Industrial Internet of Things. The DAE (Data Encryption and Decoding) after training can effectively reduce the dimensionality of the original data and extract key features that retain the core information of the data, so that the subsequent classification process has more robust fault diagnosis results for missing values. The multi-layer DBN model transforms the features into more complex representations layer by layer, supporting the complex classification task of fault diagnosis with a relatively simple structure.

[0083] Example 1

[0084] Among the various anomalies in industrial sensor networks, there are point anomalies caused by the absence of a single sensor, contextual anomalies where data does not conform to temporal variation characteristics, and cluster anomalies caused by the simultaneous failure of multiple sensors at multiple times. These anomalies often occur in conjunction with a variety of sensor absences and are closely related. Similarly, this also causes mutual interference between the two, making them difficult to distinguish. The MRFD model can effectively mine spatiotemporal information and use the features containing this correlation information for fault classification and diagnosis, minimizing the interference of missing values. To verify the spatiotemporal fusion-based fault diagnosis method for missing values ​​proposed in this invention, an experiment was designed based on TE (Tennessee-Eastman) chemical process simulation data to test the performance and practical application capability of the MRFD model.

[0085] As a classic chemical production process, the TE process includes five modules: reactor, compressor, condenser, separator, and stripping tower. A total of 41 sensors are installed in these five modules to monitor temperature, pressure, and liquid level changes during production. The entire process involves 12 control parameters. The TE dataset used in the experiment was obtained by manually adjusting the control parameters and operating conditions, followed by simulation using MATLAB. This embodiment of the invention divides the obtained data into training and testing sets for model training, and manually sets missing values ​​to simulate sensor failures in actual production processes.

[0086] The input and output of the MRFD feature extraction module are both set to 150*N, where N is the dimension of the sensor. The neurons in the intermediate layers of the feature extraction model store the core features after dimensionality reduction and are input into the DBN for fault diagnosis during the classification stage; the choice of the number of neurons is particularly important. Therefore, PCA is used for determination, and it is set to 10 neurons. During the classification stage, the depth of the hidden layers is set to 4 layers, and the input is 10 core feature variables from the feature extraction module. The number of iterations in the RBM layer is set to 15, while the number of iterations in the neural network layer is set to 100.

[0087] In this embodiment, the missing dimension of the sensor was intentionally controlled to verify the detection accuracy of MRFD under high failure conditions, and it was compared with other fault detection methods, KNN, DAE, LSTM, and STAE. The results are shown in Table 1.

[0088] Table 1. Fault diagnosis accuracy under different sensor missing dimensions.

[0089]

[0090] Experimental results show that the MRFD model, which integrates spatiotemporal correlations, achieves higher fault diagnosis accuracy than traditional methods that only consider a single correlation under the same missing dimension, demonstrating superior application performance. Furthermore, as the missing dimension of the sensor increases, the diagnostic accuracy of the MRFD model decreases much less than that of the traditional model. This is because the presence of missing values ​​often blurs the boundaries between different types of faults. The MRFD model's excellent ability to extract key features allows it to better uncover core information from the data, mitigating this impact to some extent. The MRFD model reduces the requirements of traditional fault diagnosis methods on the integrity of sensor network data, and can more accurately locate and diagnose anomalies in industrial networks with higher missing value efficiency. This demonstrates the excellent practical application potential of the spatiotemporal fusion-based robust fault diagnosis method for missing values ​​proposed in this invention.

[0091] Example 2

[0092] A fault diagnosis device based on spatiotemporal fusion for robust missing values ​​includes a processor and a memory. The memory stores program instructions, and the processor invokes the program instructions stored in the memory to cause the device to perform the following method steps:

[0093] The fault diagnosis model based on spatiotemporal fusion utilizes the temporal correlation in the data and the spatial correlation information between sensors, and uses a deep autoencoder to extract features, thereby obtaining a feature extraction model that minimizes signal reconstruction error.

[0094] The spatiotemporal correlation characteristics of time series are obtained, and a long-short-term neural network cell structure with memory of time series correlation is used, so that the feature extraction model has the ability to perform fusion analysis of spatiotemporal correlation.

[0095] Fault detection and analysis are performed based on the fused feature extraction model.

[0096] The fault diagnosis model is as follows:

[0097] The feature extraction module is a DAE structure. The DAE encoding and decoding layers are LSTM sequences connected by LSTM neurons, and the intermediate layer is a feature vector containing core feature information after dimensionality reduction. The feature extraction module is trained by minimizing the error between the original data and the reconstructed data in order to extract the most robust representation in the data to be diagnosed.

[0098] The classification module is a DBN structure, consisting of four RBM layers and one BP layer. The RBM layers map the feature vectors to different feature spaces. Only the BP layer uses label information to train the fault classifier in a supervised manner, and the error is propagated down to fine-tune the network.

[0099] Furthermore, the loss function of the feature extraction model is:

[0100]

[0101] Where r(x)=σ(Wx+b) is the reconstruction of the input signal, σ is a nonlinear function, b is the neuron bias, x is the input signal, W is the neuron weight, and λ>0 is the regularization parameter that controls the smoothness Φ(x);

[0102]

[0103] Where r(x) j-2 ), r(x j+2 ), r(x j-1 ), r(x j+1 ) is the predicted value obtained after DAE reconstruction of adjacent timestamp inputs.

[0104] The backpropagation error term of the i-th neuron in the output layer of the feature extraction model is:

[0105]

[0106] Among them, W i l This represents the weight of the i-th neuron in the output layer.

[0107] Wherein, the output of the i-th node of the DAE's output layer is o i The expected output is d i The correction error is then:

[0108] δ i =o i (1-o i )(d i -o i )

[0109] The correction error for the l-th hidden layer is calculated as follows:

[0110]

[0111] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0112] The execution entities of the processor 1 and memory 2 mentioned above can be devices with computing functions such as computers, microcontrollers, and microcontrollers. In specific implementation, the embodiments of the present invention do not limit the execution entity and can select it according to the needs of actual application.

[0113] Data signals are transmitted between memory 2 and processor 1 via bus 3, which will not be described in detail in this embodiment of the invention.

[0114] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0115] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0116] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0117] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0118] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0119] References

[0120] [1]Allison P D.Estimation of Linear Models with Incomplete Data[J].Sociological Methodology, 1987, 17: 71.

[0121] [2]Hansen M H.Sample survey methods and theory[M].1953.

[0122] [3]Little RJ A.Survey Nonresponse Adjustments for Estimates of Means[J].InternationalStatistical

[0123] Review, 1986, 54(2): 139–157.

[0124] [4]Sharpe P K,Solly R J.Dealing with Missing Values in NeuralNetwork-Based DiagnosticSystems[J / OL].Neural Computing&Applications,1995,3(2):73–77.https: / / doi.org / 10.1007 / BF01421959.

[0125] [5]Zhang A,Song S,Wang J,et al.Time series data cleaning:from anomalydetection toanomaly

[0126] repairing[C].In Proceedings of the VLDB Endowment,2017:1046–1057.

[0127] [6]Luo Y,Cai X,Zhang Y,et al.Multivariate Time Series Imputation withGenerativeAdversarial Networks[C].In Advances in Neural InformationProcessing Systems,2018:1596–1607.

[0128] [7]Luo Y,Zhang Y,Cai X,et al.E2GAN:End-to-End Generative AdversarialNetwork forMultivariateTime Series Imputation.[C].In Proceedings of theTwenty-Eighth InternationalJoint

[0129] Conference on Artificial Intelligence,2019:3094–3100.

[0130] [8]Tran L,Liu X,Zhou J,et al.Missing Modalities Imputation viaCascaded ResidualAutoencoder[C].In 2017 IEEE Conference on Computer Visionand Pattern Recognition(CVPR),2017:4971–4980.

[0131] [9]Bashir F,Wei H-L.Handling missing data in multivariate time seriesusing a vectorautoregressive model-imputation(VAR-IM)algorithm[J].Neurocomputing,2018,276:23–30.98

[0132]

[10] Asadi R,Regan A.A convolution recurrent autoencoder for spatio-temporal missing dataimputation.[J].arXiv preprint arXiv:1904.12413,2019.

[0133]

[11] Tran V-K,Nguyen L-M.Natural Language Generation for SpokenDialogue System usingRNN Encoder-Decoder Networks[C].In Proceedings of the21st Conference on ComputationalNatural Language Learning(CoNLL 2017),2017:442–451.

[0134]

[12] Cinar Y G,Mirisaee H,Goswami P,et al.Period-aware contentattention RNNs for timeseries

[0135] forecasting with missing values[J].Neurocomputing,2018,312:177–186.

[0136]

[13] Jia Y,Zhou C,Motani M.Spatio-temporal autoencoder for featurelearning in patient datawith missing observations[C].In 2017IEEEInternational Conference of BioinformaticsBiomed,2017:886–890.

[0137]

[14] Blondel V D,Guillaume J L,Lambiotte R,et al.Fast unfolding ofcommunities in largenetworks[J].Journal of Statistical Mechanics:Theory andExperiment,2008,2008(10):10008.

[15] Gia N H,Lan L N,Thi P,et al.LabelPropagation Algorithm[J],2014.

[0138]

[16] Waibel A,Hanazawa T,Hinton G,et al.Phoneme recognition usingtime-delay neuralnetworks[M].1995.

[0139]

[17] Zhang A,Song S,Wang J,et al.Time series data cleaning:fromanomaly detection toanomaly

[0140] repairing[C].In Proceedings of the VLDB Endowment,2017:1046–1057.

[0141]

[18] Chandola V,Banerjee A,Kumar V.Anomaly detection:A survey[J].ACMComputingSurveys,

[0142] 2009,41(3):15.

[0143]

[19] Hinton GA practical guide to training restricted Boltzmannmachines[J]. Momentum, 2010, 9(1):926.

[0144]

[20] Hinton GE, Osindero S, Teh Y WA fast learning algorithm for deepbelief nets[J].Neuralcomputation, 2006, 18(7):1527-1554.

[0145] Unless otherwise specified, the model numbers of the components in this embodiment of the invention are not limited.

[0146] Any device that can perform the above functions is acceptable.

[0147] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fault diagnosis method based on spatiotemporal fusion with robust missing values, characterized in that, The method includes: The fault diagnosis model based on spatiotemporal fusion utilizes the temporal correlation in the data and the spatial correlation information between sensors, and uses a deep autoencoder to extract features, thereby obtaining a feature extraction model that minimizes signal reconstruction error. The spatiotemporal correlation characteristics of time series are obtained, and a long-short-term neural network cell structure with memory of time series correlation is used, so that the feature extraction model has the ability to perform fusion analysis of spatiotemporal correlation. Fault detection and analysis based on the fused feature extraction model; The fault diagnosis model is as follows: The feature extraction module is a DAE structure. The DAE encoding and decoding layers are LSTM sequences formed by connecting LSTM neurons, and the intermediate layer is a feature vector containing core feature information after dimensionality reduction. The feature extraction module is trained by minimizing the error between the original data and the reconstructed data in order to extract the most robust representation in the data to be diagnosed. The classification module is a DBN structure, consisting of four RBM layers and one BP layer. The RBM layers map the feature vectors to different feature spaces. Only the BP layer uses label information to train the fault classifier in a supervised manner, and the error is propagated down to fine-tune the network.

2. The method for robust fault diagnosis of missing values ​​based on spatiotemporal fusion according to claim 1, characterized in that, The loss function of the feature extraction model is: ； in, For the reconstruction of the input signal, Let b be a nonlinear function, and b be the neuron bias. > 0 controls smoothness The regularization parameter; ； in, It is the predicted value obtained after DAE reconstruction of adjacent timestamp inputs.

3. The method for robust fault diagnosis of missing values ​​based on spatiotemporal fusion according to claim 2, characterized in that, The backpropagation error term of the i-th neuron in the output layer of the feature extraction model is: ； in, This represents the weight of the i-th neuron in the output layer.

4. The method for robust fault diagnosis of missing values ​​based on spatiotemporal fusion according to claim 3, characterized in that, The output of the i-th node of the DAE's output layer is The expected output is The correction error is then: ； For the l The correction error of each hidden layer is calculated using the following expression: ； Where y represents the true category of the fault data; 5. A fault diagnosis device based on spatiotemporal fusion that is robust to missing values, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1-4.

Images