Anomaly Detection Thresholding Using Information Filter Convergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for anomaly detection, such as those described by Sheinker and Zhou, face challenges in setting a suitable threshold due to the unknown optimal value and dependence on the sliding window and discretization parameters, requiring a laborious trial-and-error process.
Innovation Solution
A method and device for anomaly detection that sets a threshold based on the convergence limit value of an information filter, which is determined intuitively and adjusted for a balance between reliable detection and minimizing false alarms, using a probability density model to define the threshold as the entropy plus a small additional value.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional threshold setting methods are used (Sheinker/Zhou approaches), then anomaly detection can be performed, but the threshold determination becomes complex and requires laborious trial-and-error processes
Solution Approach 1:
The system performs self-calibration by automatically determining the threshold value through iterative application of the information filter to training data. The threshold is derived from the convergence behavior of the filter itself, eliminating the need for external manual calibration and trial-and-error processes. This self-service mechanism resolves the contradiction by making the system both reliable and simple to configure.
Solution Approach 2:
The patent implements a preliminary calibration phase where the information filter is applied to training data before actual anomaly detection. During this phase, the threshold value is pre-determined based on the filter's convergence characteristics. This preliminary action separates the complex threshold-setting task from the operational detection phase, making the system easy to use while maintaining reliability.
2Reliability
If the sliding window size N is increased to improve detection stability, then the threshold becomes more reliable, but the response time to detect anomalies increases
Solution Approach 1:
The patent employs a dynamic sliding window approach where the window size N can be adjusted based on the specific detection requirements. The system allows flexible selection of window sizes to balance between stability and response time. This dynamic adaptability resolves the contradiction by enabling users to optimize the trade-off between reliability and speed according to their specific application needs.
3Measurement precision
If the discretization step Δ is decreased to improve filter accuracy, then the anomaly detection precision improves, but the computational complexity increases
Solution Approach 1:
The patent transforms the continuous information filter into a discrete version with carefully selected parameters. By establishing specific relationships between the discretization step Δ, window size N, and threshold value, the system achieves accurate anomaly detection with manageable computational requirements. This parameter optimization resolves the contradiction by maintaining precision while controlling complexity.
Data Source
AI summary
A device and a method for detecting anomalies, including: acquiring a normal signal reflecting a normal state, determining on the basis of the normal signal a probability density ƒ which models a normal behavioral state, setting an information filter based on the probability density ƒ, the information filter being configured to converge toward a limit value L when it is applied to samples of a normal signal, while increasing its value in response to the detection of an anomaly, setting a threshold value S based on the convergence limit value, acquiring a current signal reflecting the current behavioral state, sampling the current signal in a current series of N samples, computing a result of applying the information filter to the current series of N samples, comparing the result with the threshold value S, an anomaly being detected if the result exceeds the threshold value.