Auto Reinforced Anomaly Detection via Probity Index
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data anomaly detection systems are inefficient and inaccurate, often reporting false positives and failing to verify data integrity, leading to inappropriate decisions due to the lack of context in anomaly identification.
Innovation Solution
A data anomaly detection system that includes a processor, data dissector, data mapper, and data rectifier, utilizing artificial intelligence and cognitive learning operations to validate data probity, identify anomalies, and update trends in real-time, by sorting data into wedges, determining probity scores, and removing outliers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection compares current data against entire historical span, then data coverage is comprehensive, but accuracy deteriorates due to lack of context
Solution Approach 1:
The patent segments historical data into rolling time windows (e.g., 24-hour, 7-day, 30-day periods) rather than analyzing the entire historical span at once. This segmentation preserves temporal context within each window while enabling focused comparison. The system maintains multiple segmented views of historical data to provide rich contextual information for anomaly detection without the dilution effect of aggregating all historical data into a single comparison set.
2Reliability
If data integrity checking is not performed, then processing speed is fast, but reliability deteriorates due to gaps in analysis
Solution Approach 1:
The patent performs data integrity checking and validation as preliminary actions before the main anomaly detection process. The system checks for data quality, completeness, and validity in advance, flagging or correcting issues before they affect the anomaly detection algorithm. This preliminary validation ensures reliable results without significantly impacting overall processing speed, as the integrity checks are performed efficiently on incoming data streams.
3Measurement precision
If manual analysis is used to verify data integrity, then accuracy is high, but time consumption increases
Solution Approach 1:
The patent implements automated self-service mechanisms for data integrity verification using artificial intelligence and machine learning algorithms. The system automatically detects data quality issues, validates data completeness, and verifies data consistency without manual intervention. The AI models learn from historical data patterns to identify integrity issues efficiently, providing high-accuracy verification at automated speeds, thereby eliminating the trade-off between manual verification accuracy and time consumption.
4Measurement precision
If false positives are reported as anomalies, then sensitivity is high, but reliability deteriorates leading to inappropriate decisions
Solution Approach 1:
The patent implements feedback loops where detected anomalies and their outcomes are fed back into the system to continuously refine detection accuracy. When anomalies are detected, the system analyzes the context, historical patterns, and subsequent outcomes to determine whether they were true anomalies or false positives. This feedback information is used to adjust detection thresholds, refine AI models, and improve future anomaly identification, thereby reducing false positives while maintaining high sensitivity. The feedback mechanism enables the system to learn from experience and progressively improve decision accuracy.
Data Source
AI summary
Examples of a data anomaly detection system are provided. The system may obtain a query and target data associated with a data anomaly detection requirement. The system may sort the target data into a plurality of data wedges comprising a plurality of events. The system may create a data pattern model for each of the plurality of data wedges. The system may identify a data threshold value and identify a data probity score for each of the plurality of events. The system may create a data probity index and identify a data anomaly cluster for the data pattern model. The system may generate a data anomaly detection result and initiate anomaly detection corresponding to the data anomaly detection requirement. The data anomaly detection result may include the data pattern model deficient of the data anomaly cluster relevant for resolution to the query.