Feature Selection for Anomaly Detection in Relational Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Detecting anomalies in large data sets over time is challenging due to the complexity of identifying relevant variables, which can lead to incorrect identification of issues in data analysis systems, affecting system performance and user satisfaction in applications like SEO.
Innovation Solution
A method that identifies key fields and value fields in a relational database, aggregates data based on key fields, calculates statistical values, and monitors their evolution to select features for a machine-learning model that detects anomalies, improving the model's ability to identify data anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large number of variables are analyzed in large data sets, then the completeness of anomaly detection is improved, but the complexity of the detection system increases significantly
Solution Approach 1:
The patent segments the large set of variables into two distinct groups: key fields (identifying variables) and value fields (measured variables). This segmentation allows the system to manage complexity by separating the identification function from the measurement function, enabling systematic analysis of large variable sets without overwhelming system complexity
Solution Approach 2:
The patent extracts and identifies specific key fields from the large variable set that serve as identifiers for grouping data. By extracting only the essential identifying variables (key fields) rather than analyzing all variables equally, the system reduces the effective complexity while maintaining detection accuracy through focused analysis of grouped value fields
2Reliability
If all variables in the data set are monitored, then the detection coverage is improved, but the computational resources required increase
Solution Approach 1:
The patent merges all value fields associated with the same key field into grouped data structures. By combining multiple value measurements under a single key field identifier, the system achieves comprehensive monitoring coverage while reducing computational overhead through aggregated data representation and analysis
3Device complexity
If the number of fields to be analyzed is reduced, then the system complexity is decreased, but the ability to detect actual problems may be compromised
Solution Approach 1:
The patent performs preliminary identification and grouping of variables into key fields and value fields before the actual anomaly detection process. This preliminary structuring organizes the data in advance, allowing the system to work with a manageable number of grouped key fields while preserving all necessary value field information within each group, thus maintaining detection capability with reduced operational complexity
Data Source
AI summary
Methods, systems, and computer programs are presented for selecting features for a machine-learning model configured to detect anomalies in the evolution of data over time. One method includes an operation for identifying one or more key fields and value fields from the fields in a relational database. The method also includes grouping data of the value fields based on values of the one or more key fields and calculating one or more statistical values for each group of data of the value fields. The method further includes operations for monitoring an evolution of the one or more statistical values over time, and for selecting, based on the evolution of the one or more statistical values over time, features to be used by a machine-learning model to detect anomalies in content of the relational database over time. The method also includes executing the machine-learning model to detect the anomalies.