Association Rule Mining for Outlier Detection and Record Completion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current association rule mining techniques are computationally intensive, lack scalability, and produce low-quality rules due to insufficient data, failing to detect subtle patterns and lacking accuracy and interpretability, with no unified approach to manage data aspects effectively.
Innovation Solution
An association rules-based data outlier detection and record completion system that generates rules from databases with incomplete or fuzzy data, applying them to detect outliers and complete records by identifying data dependencies, eliminating irrelevant attributes, and performing discretization and truth reinforcement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional association rule mining techniques are used, then data patterns can be discovered, but computational resources are excessively consumed and scalability is poor
Solution Approach 1:
The patent segments the data mining process into distinct phases: data preparation with attribute elimination, discretization of continuous attributes, and association rule generation. This segmentation allows each phase to be optimized independently, improving overall computational efficiency while maintaining pattern detection accuracy
Solution Approach 2:
The patent performs preliminary actions before association rule mining by eliminating irrelevant attributes and discretizing continuous attributes. This preprocessing reduces the search space and computational complexity of the subsequent rule mining process, significantly improving scalability without compromising the ability to detect subtle patterns
2Productivity
If association rules are generated from insufficient data, then rule generation can proceed, but rule quality is low and subtle patterns are missed
Solution Approach 1:
The patent implements a self-service mechanism where the system automatically evaluates generated association rules against the database and iteratively refines them. This self-correction process improves rule quality by identifying and eliminating low-quality rules while preserving those that accurately represent subtle patterns in the data
Solution Approach 2:
The patent incorporates feedback mechanisms where generated rules are evaluated based on support and confidence metrics, and this evaluation feedback is used to refine the mining process. Rules that fail to meet quality thresholds are eliminated or refined, ensuring that only high-quality rules representing genuine patterns are retained
3Loss of time
If data preparation is not performed, then processing time is reduced, but irrelevant attributes reduce rule quality and interpretability
Solution Approach 1:
The patent extracts and eliminates irrelevant attributes from the dataset before association rule mining. By identifying and removing attributes that do not contribute to meaningful patterns, the system improves rule interpretability and quality without requiring excessive processing time, as the attribute elimination process is computationally efficient
Data Source
AI summary
The present disclosure comprises systems and methods to mine association rules from a dataset provided in the input. This comprises methodologies for optimizing the dataset for efficient computation of association rules, other than methodologies for handling partially true data. Moreover, it includes methodologies for evaluating the mining process automatically and removing uninteresting rules. In addition, it includes methodologies for integrating experts in the evaluation of the rules. Finally, methodologies to automatically detect outliers, correct outliers, update truth values and complete missing data in the original dataset. The overall methodology is completely automated and provides numerous tuning parameters to fit most of the use cases, including a default value for each of the tuning parameters to simplify its usage.