Autoencoder Anomaly Detection for Irregular Sequential Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting abnormal data in sequentially collected, mutually associated data are not universal and fail to detect anomalies in various types of data, such as those not following a normal distribution or containing fierce vibration signals.
Innovation Solution
A method using autoencoders to process detection data sets, trained with data having a regular pattern of variation, to identify abnormal data by determining differences between input and output data, thereby detecting anomalies in all types of sequentially collected, mutually associated data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional detection methods (3δ principle, small-wave smoothing, speed constraint smoothing) are used, then detection simplicity is maintained, but detection accuracy deteriorates for certain data types (non-normal distribution, fierce vibration, square-wave data)
Solution Approach 1:
The patent replaces traditional mechanical/mathematical detection methods (3δ principle, smoothing methods) with a neural network-based autoencoder system. The autoencoder learns the normal patterns of sequentially collected data during training, then identifies abnormalities by detecting deviations from these learned patterns, achieving universal applicability across different data types without requiring method-specific adjustments.
Solution Approach 2:
The patent transforms the detection approach by changing from fixed threshold-based parameters (like 3δ) to adaptive parameters learned through neural network training. The autoencoder dynamically adjusts its internal parameters (weights and biases) during training to match the specific characteristics of the input data, enabling accurate detection across varying data distributions and patterns.
2Adaptability or versatility
If universal detection method is implemented using autoencoder, then adaptability to all data types is improved, but device complexity increases
Solution Approach 1:
The patent implements a universal detection system using autoencoder that can handle all types of sequentially collected data (non-normal distribution, fierce vibration signals, square-wave data, etc.) with a single unified approach. The autoencoder's ability to learn arbitrary patterns during training makes it adaptable to diverse data types without requiring separate detection methods for each data category.
Solution Approach 2:
The autoencoder performs self-training by learning the normal patterns of the input data during a training phase. The system automatically adapts to the specific characteristics of each data type through unsupervised learning, eliminating the need for manual parameter tuning or method selection based on data type. The model serves itself by automatically configuring its detection thresholds and patterns.
3Measurement precision
If more training data is used to improve autoencoder performance, then detection precision is improved, but data processing time increases
Solution Approach 1:
The patent performs preliminary training of the autoencoder model using historical normal data before actual abnormal detection begins. This pre-training phase allows the system to learn and store the normal patterns of data in advance, so that during operational detection, only forward propagation through the trained network is needed, which is computationally efficient. The time-consuming learning process is separated from the real-time detection process.
Data Source
AI summary
In an embodiment, a method includes: using at least two items of sequentially collected, mutually associated data to create at least two detection data sets, each including a first number of items of sequentially collected data in the at least two items of data; using an autoencoder to process the at least two detection data sets, to output result data sets respectively corresponding to the at least two detection data sets, the first number being equal to the number of neurons in an input layer of the autoencoder, and the autoencoder being trained using data having a regular pattern of variation identical to the at least two items of data; and determining, as abnormal data, data which does not have the regular pattern of variation in the at least two items of data, based upon the at least two detection data sets and the result data sets corresponding thereto.