Novelty Detection Using Intermediate Layer Comparison in Autoencoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing novelty detection methods using deep learning neural networks do not fully exploit information from intermediate layers, leading to a loss of valuable information and inadequate detection of novelty, especially in scenarios with imbalanced data sets where normal data is abundant and abnormal data is scarce.
Innovation Solution
The method involves a deep learning neural network model with an autoencoder that processes inputs through both encoder and decoder layers multiple times, generating intermediate encoded inputs and reconstructed outputs, which are then compared to quantify novelty using a novelty score that leverages information from hidden spaces along the projection pathway.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an input is processed through a deep learning neural network to obtain an output for novelty detection, then the detection process can be performed, but valuable information from intermediate layers is lost leading to inadequate detection accuracy
Solution Approach 1:
The patent segments the deep learning neural network into multiple intermediate layers between the input and output. Instead of processing the input through the entire network to a single output, the system extracts features from multiple intermediate layers (e.g., first intermediate output, second intermediate output) and uses these segmented representations for novelty detection. This segmentation preserves valuable information that would otherwise be lost in the compression process.
Solution Approach 2:
The patent introduces an additional dimension to novelty detection by incorporating intermediate layer outputs alongside the final network output. The novelty detection mechanism now operates in a multi-dimensional feature space that includes both the final output and intermediate representations. This dimensional expansion allows the system to capture nuanced patterns and relationships that single-output approaches miss, thereby improving detection accuracy without information loss.
2Reliability
If existing novelty detection methods are used, then the basic detection function is achieved, but the methods fail to fully exploit information from hidden spaces leading to suboptimal performance
Solution Approach 1:
The patent makes the deep learning neural network multi-functional by using it for both feature extraction and novelty detection simultaneously. The intermediate layers serve dual purposes: they act as feature extractors for the input data while also providing direct inputs to the novelty detection mechanism. This universal utilization ensures that no information is wasted, as every layer contributes to both representation learning and anomaly detection.
Solution Approach 2:
The patent implements a feedback mechanism where the intermediate layer outputs are fed back into the novelty detection process. The system continuously refines its detection by using the information from intermediate layers to adjust and improve the novelty assessment. This feedback loop ensures that hidden space information is fully exploited to enhance detection reliability.
3Device complexity
If a simple single-output neural network is used, then the model complexity is low, but the novelty detection performance is insufficient especially with imbalanced datasets
Solution Approach 1:
The patent segments the neural network into multiple intermediate layers that each perform feature extraction at different levels of abstraction. This segmentation increases model complexity in a controlled manner, allowing the system to capture complex patterns in imbalanced datasets. Each intermediate layer contributes specialized feature representations that collectively enable high-precision novelty detection.
Solution Approach 2:
The patent adds another dimension to the neural network architecture by incorporating multiple intermediate output layers. This dimensional expansion allows the model to process and represent information in a richer, multi-level feature space. The enhanced dimensionality enables the system to distinguish between normal and abnormal patterns more effectively, improving detection precision without excessive complexity.
Data Source
AI summary
The disclosed technology generally relates to novelty detection and more particularly to novelty detection methods using a deep learning neural network and apparatuses and non-transitory computer-readable media configured for performing the methods. In one aspect, a method for detecting novelty using a deep learning neural network model comprises providing a deep learning neural network model. The deep learning neural network model comprises an encoder comprising a plurality of encoder layers and a decoder comprising a plurality of decoder layers. The method additionally comprises feeding a first input into the encoder and successively processing the first input through the plurality of encoder layers to generate a first encoded input, wherein successively processing the first input comprises generating a first intermediate encoded input from one of the encoder layers prior to generating the first encoded input. The method additionally comprises feeding the first encoded input from the encoder into the decoder and successively processing the first encoded input through the plurality of decoder layers to generate a first reconstructed output. The method additionally comprises feeding the first reconstructed output from the decoder as a second or subsequent input into the encoder and successively processing the first reconstructed output through the plurality of encoder layers, wherein successively processing the first reconstructed output comprises generating a second intermediate encoded input from the one of the encoder layers. The method further comprises detecting a novelty of the original input based on a comparison of the first intermediate encoded input and the second intermediate encoded input.