Relational Database Frequent Itemset Mining with Metric Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data mining techniques in relational database systems require repeated analysis of historical data to extract metric values for association rules, leading to increased costs, database access, network communications, and processing time.
Innovation Solution
A system that determines frequent item sets and aggregates metric values simultaneously within a relational database system, using methodologies like prefix trees and bitmaps to count occurrences and calculate aggregate metrics in a single pass through the transaction data, reducing the need for post-processing and optimizing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If repeated analysis of historical data is performed to extract metric values for association rules, then complete metric extraction is achieved, but costs and processing time increase
Solution Approach 1:
The system performs preliminary aggregation of metric values during the frequent itemset mining process itself. Instead of extracting metric values after association rules are formed, the system accumulates metric values (such as revenue, quantity, profit) in advance during the same pass through the data that identifies frequent itemsets, eliminating the need for repeated data analysis
Solution Approach 2:
The system merges the frequent itemset mining process with the metric aggregation process into a single unified operation. Both tasks are performed simultaneously during one pass through the transaction data, combining what were previously separate sequential operations into a concurrent process that reduces total processing time
2Reliability
If repeated analysis of historical data is performed to extract metric values, then accurate association rules are formed, but database access increases
Solution Approach 1:
The system performs preliminary aggregation of metric values during the frequent itemset mining process itself. Instead of extracting metric values after association rules are formed, the system accumulates metric values (such as revenue, quantity, profit) in advance during the same pass through the data that identifies frequent itemsets, eliminating the need for repeated data analysis
3Measurement precision
If repeated analysis of historical data is performed to extract metric values, then complete metric extraction is achieved, but network communications increase
Solution Approach 1:
The system merges the frequent itemset mining process with the metric aggregation process into a single unified operation. Both tasks are performed simultaneously during one pass through the transaction data, combining what were previously separate sequential operations into a concurrent process that reduces total processing time
4Productivity
If single pass through transaction data is used to determine frequent item sets and aggregate metrics, then processing time is reduced, but system complexity increases
Solution Approach 1:
The system implements a universal data processing framework that handles both frequent itemset mining and metric aggregation through the same core mechanisms. The prefix-tree construction and bitmap operations serve dual purposes: identifying frequent itemsets while simultaneously accumulating metric values, making the system multi-functional without requiring separate processing paths
Data Source
AI summary
Systems, methods, and other embodiments are disclosed for performing data mining. In one embodiment, transaction records are read one at a time. Each transaction record represents a transaction for at least one item and includes an item identifier and a metric value for the item. The number-of-occurrences of at least one candidate item set in the transaction records are counted to generate a total count for the candidate item set. The candidate item set includes one or more items. As the counting proceeds, at least one aggregate metric value associated with the candidate item set is accumulated by summing metric values across the number-of-occurrences for each item represented in the candidate item set. A determination is made as to whether the candidate item set is a frequent item set in the transaction records by comparing the total count to a threshold value.