Frequent Itemset Mining Using Ubiquitousness Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frequent itemset mining algorithms face challenges with high computational complexity and memory requirements due to scanning all item combinations in large datasets, often leading to non-convergence or unmeaningful results, especially with dense transactions and frequent items that do not contribute to association rules.
Innovation Solution
The introduction of a 'ubiquitousness' parameter to filter out items with high frequency occurrences, reducing the number of scans and computational complexity by ignoring items that do not contribute to meaningful rules, allowing for a tradeoff between support, confidence, and ubiquitousness constraints to derive optimal association rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If all item combinations are scanned to ensure comprehensive association rule discovery, then the completeness of mined rules is improved, but the computational complexity and memory requirements increase significantly
Solution Approach 1:
The patent extracts and removes items with high ubiquitousness (frequently occurring items that do not contribute to meaningful association rules) from the dataset before applying the Apriori algorithm. This extraction reduces the search space from all possible item combinations to only those involving non-ubiquitous items, thereby decreasing computational complexity while preserving the completeness of meaningful rule discovery.
Solution Approach 2:
The patent performs a preliminary action by calculating the ubiquitousness parameter for all items and filtering out high-ubiquitousness items before the main FIM process. This preliminary filtering step prepares the data in advance to avoid unnecessary computations during the association rule mining phase, reducing overall computational burden.
2Manufacturing precision
If all item combinations are scanned to ensure comprehensive rule discovery, then the completeness of mined rules is improved, but the memory requirements increase significantly
Solution Approach 1:
The patent extracts items with high ubiquitousness values and removes them from the transaction dataset before processing. This extraction reduces the number of items that need to be tracked and stored during the Apriori algorithm execution, thereby reducing memory requirements while maintaining the ability to discover all meaningful association rules.
3Manufacturing precision
If frequent items are included in the analysis to ensure comprehensive coverage, then the coverage of mined rules is improved, but the running time increases significantly
Solution Approach 1:
The patent identifies and extracts items with high ubiquitousness (frequent items) from the dataset before applying the Apriori algorithm. By removing these frequent items that do not contribute to meaningful association rules, the patent reduces the number of candidate itemsets that need to be evaluated, thereby significantly reducing the running time while maintaining coverage of all meaningful rules.
Solution Approach 2:
The patent performs preliminary filtering of frequent items based on ubiquitousness threshold before the main mining process. This advance preparation prevents the algorithm from wasting time generating and evaluating candidate itemsets that would inevitably be pruned due to their frequent occurrence, thus optimizing running time.
4Manufacturing precision
If the ubiquitousness threshold is lowered to include more items, then the completeness of mined rules is improved, but the computational complexity increases
Solution Approach 1:
The patent introduces a dynamic ubiquitousness threshold parameter that can be adjusted based on the specific dataset and mining objectives. This dynamic parameter allows the system to adaptively balance between completeness of rule discovery and computational complexity, enabling users to optimize the threshold according to their specific needs rather than using a fixed value.
Data Source
AI summary
The present disclosure describes methods, systems, and computer program products for improving frequent itemset mining. One computer-implemented method includes receiving, by operation of a computer system, a dataset of multiple transactions, each of the multiple transactions including one or more items; determining, by operation of the computer system, a ubiquitousness parameter of an item representing a frequency of occurrences of the item in the dataset; filtering, by operation of the computer system, out from the dataset one or more items that have ubiquitousness parameters larger than a ubiquitousness threshold; identifying, by operation of the computer system, one or more association rules that satisfy a confidence constraint based on the filtered dataset; and outputting, by operation of the computer system, the one or more association rules that satisfy the confidence constraint.