Autoencoder Neural Network for Recovering Missing Sensor Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Industrial systems like SCADA and IoT face reliability issues due to faulty or inaccessible sensors, leading to missing data that hinders AI algorithm performance and can result in catastrophic failures, especially in critical industries like power generation and HVAC systems.
Innovation Solution
A neural network with an autoencoder architecture is trained in two stages to recover missing sensor data by learning efficient encoding and decoding processes, even in the presence of missing data, using a self-supervised approach that transforms unsupervised learning into a supervised method to handle noise and interdependencies between sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sensors are deployed in industrial systems for data collection, then monitoring capability is improved, but sensor inaccessibility and data missing occur leading to reduced reliability
Solution Approach 1:
The system performs preliminary actions by training the autoencoder model on complete sensor data before deployment. The model learns the relationships between sensors in advance, enabling it to predict missing values when sensors become inaccessible during operation.
Solution Approach 2:
The autoencoder creates a virtual copy of the missing sensor data by predicting values based on patterns learned from other sensors. This copied data allows the system to continue operating as if the physical sensor were still functional.
2Measurement precision
If AI algorithms use a fixed set of sensor inputs, then algorithm performance is improved, but missing sensor data breaks operation
Solution Approach 1:
The autoencoder serves as an intermediary between the physical sensors and the AI algorithms. When sensors are inaccessible, the autoencoder provides predicted values that mediate between the missing data and the algorithm's requirements, allowing continuous operation without modification to the AI algorithms.
Solution Approach 2:
The system changes the parameter state of missing sensor data from 'missing' to 'predicted' by using the autoencoder. This parameter transformation allows the fixed input requirements of AI algorithms to be satisfied even when physical sensors are inaccessible.
3Ease of operation
If sensor data is used directly in analytical formulas, then processing simplicity is improved, but missing data creates problems
Solution Approach 1:
The system implements self-service by having the autoencoder automatically detect and fill missing sensor values without human intervention. The model serves itself by identifying gaps in the data and generating appropriate predictions, maintaining processing simplicity while ensuring data completeness.
Data Source
AI summary
A computer-implemented method of training an autoencoder to recover missing data is provided. The autoencoder includes an encoder for encoding its inputs into a latent space and a decoder for decoding the encodings from the latent space. The method comprises creating a first training set including a valid data set of multiple dimensions, and training the encoder and the decoder in a first training stage using the first training set to reduce a difference between the valid data set provided to the encoder and a data set decoded by the decoder. The method further comprises creating a second training set comprising an invalid data set, and training the encoder in a second training stage using the second training set to reduce a difference between encodings of valid data instances and encodings of their corresponding invalid data instances.