Data Grouping for Accurate Anomaly Detection Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for detecting anomalies in data entries are prone to human error, lack standardization, and are inefficient due to the overwhelming quantity of data, leading to high false positives and false negatives, and inaccurate performance metrics.
Innovation Solution
A centralized system that groups data entries based on shared selection criteria, generates performance metrics, and optionally uses machine learning to identify anomalies, reducing computational load and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review methods are used to detect anomalies in data entries, then reviewers can examine each data entry individually, but the process becomes slow, error-prone, and unable to handle large quantities of data efficiently
Solution Approach 1:
The patent segments the large dataset into smaller groups based on shared selection criteria values. By dividing the data into manageable segments that can be processed in parallel, the system maintains high anomaly detection accuracy while significantly improving processing speed and productivity.
Solution Approach 2:
The patent replaces the manual mechanical review process with an automated computer-implemented system. The processors automatically group data, compare values, and detect anomalies, eliminating human error and handling large datasets efficiently while maintaining or improving detection accuracy.
2Reliability
If reviewers manually group and compare data entries to detect anomalies, then they can analyze data in context, but human error increases and standardization becomes difficult
Solution Approach 1:
The patent implements a universal automated system that performs multiple functions: grouping data by selection criteria, comparing data values, detecting anomalies, and generating performance metrics. This single standardized system handles all data processing tasks consistently, eliminating variability between different reviewers while managing complexity through integration.
Solution Approach 2:
The patent changes the processing parameters from manual human analysis to automated computational analysis. By transforming the data processing approach into standardized computational operations with defined algorithms and criteria, the system achieves consistent and reliable anomaly detection across all datasets.
3Measurement precision
If all data entries are analyzed individually for anomalies, then comprehensive coverage is achieved, but the computational load becomes overwhelming and processing time increases
Solution Approach 1:
The patent segments the comprehensive data analysis task into smaller grouping operations based on shared selection criteria. This segmentation allows the system to achieve comprehensive anomaly detection coverage across all data entries while reducing processing time by working with organized, smaller groups rather than analyzing every entry individually in isolation.
Solution Approach 2:
The patent performs preliminary grouping and organization of data by selection criteria before conducting anomaly detection. This preliminary action structures the data in advance, making the subsequent anomaly detection process more efficient and reducing the overall processing time while maintaining comprehensive coverage.
4Measurement precision
If performance metrics are calculated from ungrouped data, then all data points are included, but the metrics become inaccurate and difficult to interpret
Solution Approach 1:
The patent segments the data into grouped rows based on shared selection criteria before calculating performance metrics. This segmentation allows for more accurate and interpretable metrics by considering the contextual relationships within groups, while the automated grouping process manages the organizational complexity systematically.
Data Source
AI summary
A computer-implemented method of accessing, by one or more processors, database tables that include a first row storing a first set of data that includes selection criteria values and first data values and a second row storing a second set of data that includes the selection criteria values and second data values. After determining the first and second sets of data share the selection criteria, the method groups the first and second row to generate a grouped row that indicates a comparison of data based on certain data formats. The method iteratively performs the grouping across the database tables to generate a plurality of grouped rows which are incremented by a counter value to reflect the total number of grouped rows. The method generates a performance metric based on the counter value incremented.