Sampling-Based Association Rule Mining for Large Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current association rule mining techniques face challenges in setting minimum support and confidence thresholds, require significant computational effort, and are inefficient due to exponential growth of frequent itemsets, with existing sampling methods either inaccurate or computationally intensive.
Innovation Solution
The development of sampling-based methods and systems that heuristically generate minimum support and confidence thresholds, utilizing random, systematic, stratified, and heuristic sampling, as well as classification methodologies to improve the efficiency of association rule mining, including the use of the FAST algorithm for accurate and resource-efficient discovery of frequent itemsets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Apriori algorithm is used to mine association rules, then comprehensive rule extraction is achieved, but computational effort increases significantly on large datasets
Solution Approach 1:
The patent applies partial action by using sampling techniques to process only a subset of transactions rather than the entire dataset. This allows the system to extract association rules with acceptable accuracy while significantly reducing computational effort and processing time on large datasets.
Solution Approach 2:
The patent segments the large transaction dataset into multiple smaller samples that can be processed independently. This segmentation enables parallel processing and reduces the computational burden on single processing units while maintaining comprehensive rule extraction across the entire dataset.
2Quantity of substance
If FP-growth algorithm is used to mine association rules, then memory efficiency is improved, but the number of extracted rules decreases compared to Apriori
Solution Approach 1:
The patent uses sampling to process a representative subset of transactions, which maintains memory efficiency while recovering the number of extracted rules through statistical generalization from the sample to the full dataset.
Solution Approach 2:
The patent performs preliminary sampling and analysis to identify promising itemsets and patterns before applying the FP-growth algorithm. This preliminary action guides the mining process to focus on relevant patterns, increasing the number of useful rules extracted while maintaining memory efficiency.
3Loss of time
If random sampling is used to speed up frequent itemset mining, then processing time is reduced, but accuracy in representing the actual population decreases
Solution Approach 1:
The patent implements feedback mechanisms where the sampling process is iteratively refined based on the results obtained. The system analyzes patterns in the sampled data, adjusts sampling parameters, and re-samples to improve accuracy while maintaining reduced processing time compared to exhaustive methods.
Solution Approach 2:
The patent uses dynamic sampling strategies where the sampling rate and selection criteria adapt based on the characteristics of the data being processed. This dynamic approach ensures that critical patterns are captured in the sample while maintaining efficient processing speeds.
4Measurement precision
If heuristic sampling is used to improve accuracy over random sampling, then representation accuracy increases, but computational resources required increase excessively
Solution Approach 1:
The patent applies partial heuristic sampling by using simplified heuristic criteria on a subset of data rather than applying complex heuristic algorithms to the entire dataset. This approach maintains acceptable accuracy while avoiding excessive computational resource requirements.
Data Source
AI summary
Systems, methods, and computer-readable code stored on a non-transitory media for mining association rules include determining a minimum support threshold and a minimum confidence threshold for association rule mining; determining a sampling model; sampling transactions from a transaction dataset; mining association rules from the sampled transactions; and transmitting mined association rules.