Diverse Anomalous Subset Discovery Using Penalized Intersections
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for anomalous subset discovery face challenges in handling large search spaces in high-dimensional datasets, leading to computational inefficiencies and redundancy in subset discovery, particularly in top-k subset discovery, where obtaining multiple anomalous subsets becomes increasingly difficult due to exponential growth in the number of potential subsets, and existing techniques fail to adequately address diversity and redundancy at the record level.
Innovation Solution
The proposed method employs a penalized intersection approach using a Multi-dimensional Subset Scan (MD-Scan) with a regularization component to iteratively penalize overlapping records, ensuring diversity and minimizing redundancy by controlling the diversity score through a user-defined threshold, enabling exploration of larger search spaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional subset discovery methods are used to explore the search space, then all potential subsets can be evaluated, but the computational complexity grows exponentially and redundancy increases
Solution Approach 1:
The patent extracts and penalizes only the overlapping records between candidate subsets and previously selected subsets, rather than evaluating all subsets. By identifying and removing redundant records through the penalized intersection operation, the method focuses computational effort only on novel, non-redundant portions of the search space, thereby reducing exponential complexity while maintaining discovery accuracy.
Solution Approach 2:
The patent performs preliminary penalization of overlapping records before final subset selection. By pre-computing and marking redundant records with penalties based on their intersection with previously selected subsets, the method eliminates redundancy in advance, allowing subsequent subset evaluation to proceed efficiently without re-examining already-processed records.
2Adaptability or versatility
If multiple anomalous subsets are discovered in top-k subset discovery, then more diverse patterns can be identified, but the computational cost and difficulty increase exponentially
Solution Approach 1:
The patent implements a feedback mechanism where each newly selected subset feeds back into the system by penalizing its overlapping records. This penalty information is incorporated into the scoring function for subsequent candidate subsets, creating a dynamic feedback loop that guides the discovery process toward diverse, non-redundant patterns while maintaining computational efficiency through iterative refinement.
Solution Approach 2:
The patent transforms the static subset discovery problem into a dynamic process where the search space evolves with each selected subset. The penalized intersection creates a dynamically changing landscape where previously selected subsets actively influence future selections by modifying the scores of candidate subsets, enabling efficient exploration of diverse patterns without exhaustive search.
3Ease of manufacture
If existing techniques are used for subset discovery, then implementation is simpler, but they fail to adequately address diversity and redundancy at the record level
Solution Approach 1:
The patent applies local quality by implementing record-level penalization rather than uniform subset-level processing. Each record receives a personalized penalty based on its specific overlap with previously selected subsets, allowing fine-grained control over redundancy at the individual record level. This localized approach maintains implementation simplicity while achieving precise control over record-level diversity.
Data Source
AI summary
One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to automated control for a physical system with generic forecasting models. The computer-implemented system can comprise a memory that can store computer executable components. The computer-implemented system can further comprise a processor that can execute the computer executable components stored in the memory, wherein the computer executable components can comprise a discovery component that obtains a candidate anomalous subset of a dataset, a scoring component that computes a diversity score of the candidate anomalous subset relative to selected subsets of the dataset, and a selection component that selects the candidate anomalous subset based on the diversity score. Furthermore, the level of diversity between subsets can be controlled by a user. Moreover, records can be penalized to enable search space exploration and mitigate redundancy at the record level.