Information Processing Apparatus for Efficient Item Set Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting sets of items that co-occur frequently in transaction data are computationally complex, especially as the number of items increases, due to the need to evaluate every combination for commonality.
Innovation Solution
An information processing apparatus and method that deletes sets and elements not meeting threshold criteria, groups remaining sets by commonality, and outputs lists of elements within these groups, reducing computational complexity by focusing on associations rather than all possible combinations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If every different combination of items is confirmed to satisfy extraction conditions, then extraction precision is improved, but computational complexity increases
Solution Approach 1:
The patent segments the extraction process into two distinct phases: (1) generating candidate item sets using efficient algorithms like Apriori that leverage item frequency and co-occurrence patterns, and (2) verifying these candidates against extraction conditions. This segmentation avoids exhaustive enumeration of all possible combinations while maintaining extraction precision by systematically generating only promising candidates.
Solution Approach 2:
The patent performs preliminary filtering of items based on their individual frequencies and co-occurrence patterns before full verification. By pre-processing the data to identify frequently occurring items and their associations, the system reduces the search space for subsequent extraction, thereby lowering computational complexity while preserving extraction accuracy.
2Adaptability or versatility
If the number of items increases, then the range of extracted knowledge is improved, but the number of combinations increases and computational complexity increases
Solution Approach 1:
The patent employs dynamic thresholding where minimum support and confidence thresholds are adjusted based on the scale of data and number of items. This dynamic adaptation allows the system to handle varying numbers of items efficiently, maintaining extraction quality while scaling to larger item sets by adapting the strictness of extraction conditions to the problem size.
Solution Approach 2:
The patent changes key parameters such as minimum support threshold and confidence threshold based on the number of items in the dataset. When the number of items increases, the system adjusts these parameters to maintain computational feasibility while still extracting meaningful knowledge patterns, thereby scaling the system's adaptability to different data sizes.
3Productivity
If efficient algorithms for graph manipulation are used, then process efficiency is improved, but no technique exists for extracting sets of items based on commonality of items
Solution Approach 1:
The patent introduces an intermediary data structure (item co-occurrence matrix or graph representation) that mediates between raw transaction data and final extracted patterns. This intermediary structure enables efficient querying of item commonality relationships while maintaining the ability to extract patterns based on item commonality, thus bridging the gap between efficiency and capability.
Solution Approach 2:
The patent transforms the extraction problem from a combinatorial search through item combinations into a graph traversal or matrix operation problem by representing items as nodes and co-occurrences as edges or matrix entries. This dimensional transformation enables the use of efficient graph algorithms while preserving the ability to extract patterns based on item commonality relationships.
Data Source
AI summary
In an information processing apparatus, a first deletion unit deletes, from sets stored in a storage unit, sets having less than a threshold number of included elements and elements existing in only sets whose number is less than a threshold number of occurrences. A grouping unit generates a group of sets associated with each other in terms of commonality of elements out of the sets remaining in the storage unit. A second deletion unit deletes, with respect to each generated group, sets having less than the threshold number of included elements and elements existing in only sets whose number is less than the threshold number of occurrences, from the sets belonging to the group. An output unit outputs a list of elements included in the sets remaining in each group where there are no sets or elements that need to be deleted.