AI Anomaly Detection Using Conformity Index Feature Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods face challenges in efficiently detecting significant deviations while minimizing computational load and false positives, particularly in the context of increasing data volumes and variable complexity.
Innovation Solution
A method involving training an AI model using reference data streams to compute non-conformity features with conformity indices, generating augmented behavioral datasets, and applying impact factors to enhance detection accuracy and reduce dimensionality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the threshold for anomaly detection is lowered to increase sensitivity, then detection sensitivity improves, but false positives increase and efficiency decreases
Solution Approach 1:
The patent transforms the threshold parameter into a dynamic conformity index that adapts to the distribution of behavioral features. By computing conformity indices based on the statistical properties of reference data streams, the system automatically adjusts sensitivity thresholds without manual intervention, resolving the contradiction between detection sensitivity and efficiency.
Solution Approach 2:
The system performs self-calibration by automatically computing conformity indices from reference data streams and using these indices to dynamically adjust detection thresholds. This self-service mechanism eliminates the need for manual threshold tuning, allowing the system to maintain optimal sensitivity while avoiding false positives through automated adaptation to normal behavior patterns.
2Measurement precision
If the amount of data and variables examined is increased to improve detection accuracy, then detection accuracy improves, but computational load increases and real-time performance deteriorates
Solution Approach 1:
The patent extracts and selects only the most relevant behavioral features for anomaly detection by computing conformity indices for each feature and filtering out those with low conformity. This extraction process reduces the dimensionality of data examined while maintaining detection accuracy, directly addressing the contradiction between accuracy and computational load.
Solution Approach 2:
The system segments the anomaly detection process into distinct stages: computing conformity indices for individual features, aggregating these indices into overall conformity scores, and making detection decisions based on these aggregated scores. This segmentation allows the system to process large amounts of data efficiently by breaking down the computational task into manageable, parallelizable operations.
Data Source
AI summary
The invention relates to a method for detecting anomalies in a data stream. The method including a training step comprising, based on at least one reference data stream, computing a value of at least one predetermined non-conformity feature, thereby obtaining a conformity index; for each reference data stream, computing a reference behavioral dataset including, for at least one predetermined behavioral feature, a value of said behavioral feature computed based on said reference data stream, The training step also includes, for each reference data stream, generating a respective reference augmented behavioral dataset including the respective reference behavioral dataset and, for each non-conformity feature, a respective deviation value equal to zero; and training an artificial intelligence model based on each reference augmented behavioral dataset, each reference augmented behavioral dataset being associated with information indicative of the absence of anomaly in the corresponding reference data stream.