Rolling Filter Bitmaps for Dynamic Association Rule Mining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Association Rule Mining (ARM) algorithms face challenges in efficiently processing dynamic data, leading to high computation costs and inefficiencies, especially when data changes over time, making them unsuitable for real-time applications.
Innovation Solution
The implementation of a computer-implemented method using a rolling filter bitmapping scheme, such as a Bloom filter, to generate filtering bitmaps for dynamic data, allowing for efficient identification of frequent itemsets without rescanning the entire database, thereby improving performance and stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional ARM algorithms rescan the entire database when data changes, then completeness of data processing is maintained, but computation cost and processing time increase significantly
Solution Approach 1:
The patent pre-processes data into a compact representation (bitmask or hash table) before changes occur. When data changes, only the affected portions need updating rather than rescanning the entire database, thus maintaining completeness while reducing computation cost.
Solution Approach 2:
The patent extracts only the necessary data representations into a condensed format (bitmask per variable or hash table entries). This extraction allows the system to process changes efficiently by working with the extracted representations rather than the full database, reducing the computational burden during updates.
2Reliability
If conventional ARM algorithms process dynamic data by rescanning the entire database, then data completeness is ensured, but processing time increases making real-time applications infeasible
Solution Approach 1:
The system performs preliminary processing by maintaining updated data representations (bitmasks or hash tables) that reflect current data states. When changes occur, only the affected representations are updated, enabling fast processing without time-consuming full database rescans, thus achieving real-time performance while maintaining data completeness.
Solution Approach 2:
The patent employs dynamic data structures (bitmasks that can be updated incrementally, hash tables that support efficient insertions and deletions) to adapt to changing data. These dynamic structures allow the system to respond to data changes efficiently without requiring complete reprocessing, thereby reducing processing time while ensuring data completeness.
3Measurement precision
If conventional ARM algorithms generate frequent itemsets through multiple iterations, then mining accuracy is improved, but system resources are exhausted
Solution Approach 1:
The patent creates a simplified copy or representation of the data (bitmask per variable, hash table of itemsets) that captures the essential information needed for mining. This copy can be processed more efficiently than the full database, allowing multiple iterations for accuracy improvement while consuming fewer system resources since the working data structure is much smaller.
Solution Approach 2:
The patent changes the representation parameters of the data from full transaction records to condensed forms (bitmasks or hash tables). This parameter transformation maintains the essential mining information while significantly reducing the data volume that needs to be processed during iterative refinement, thus improving mining accuracy without exhausting system resources.
4Ease of manufacture
If conventional ARM algorithms use traditional data structures for processing, then ease of implementation is maintained, but adaptability to dynamic data is poor
Solution Approach 1:
The patent transforms the data representation parameters into formats better suited for dynamic data (bitmasks for efficient updates, hash tables for flexible access). These parameter changes enhance adaptability to dynamic data while maintaining reasonable implementation ease through standard programming techniques and well-known data structure operations.
Solution Approach 2:
The patent replaces traditional mechanical data processing approaches (full database scans, complex iterative algorithms on large datasets) with more elegant solutions using bit-level operations and hash-based structures. This substitution maintains implementation feasibility through standard computational operations while dramatically improving adaptability to dynamic data scenarios.
Data Source
AI summary
Embodiments of the present invention provide systems and methods for associating rule mining. By way of example, embodiments obviate or mitigate performance and stability problems associated with conventional association rule mining approaches, by proposing concepts implementing association rule mining using rolling filtering bitmaps over dynamic data.