Anomaly Detection in Storage Systems via Community Access Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Operational teams in enterprise organizations face challenges in handling vast amounts of data across on-premise and cloud-based storage sources, with difficulties in differentiating between legitimate and suspicious user access patterns, leading to potential false alarms and inefficient data security.
Innovation Solution
A system and method for detecting anomalies in user access patterns by forming communities of users based on shared access to entities in a distributed storage system, using a community engine to normalize access rates and flag suspicious behavior based on collective community activity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If individual user access patterns are monitored against their own history, then anomaly detection capability is improved, but false alarm rate increases
Solution Approach 1:
The patent merges individual user access patterns with community-level access patterns to form a composite baseline. By combining individual history with collective community behavior, the system achieves more reliable anomaly detection that reduces false alarms while maintaining detection sensitivity.
Solution Approach 2:
The community access pattern serves as an intermediary between individual user behavior and organizational security policies. This intermediary layer contextualizes individual anomalies by comparing them against peer group behavior, filtering out false positives caused by legitimate but unusual individual actions.
2Reliability
If community-based normalization is applied to user access rates, then false alarm reduction is achieved, but computational complexity increases
Solution Approach 1:
The system segments users into communities based on their access patterns and relationships. This segmentation allows computation to be distributed across community groups rather than requiring organization-wide analysis, reducing overall computational complexity while maintaining normalization accuracy.
Solution Approach 2:
The system pre-computes community access patterns and stores them for later comparison. By performing the computationally intensive community analysis in advance and caching results, the system reduces real-time computational requirements while maintaining reliable false alarm reduction.
3Reliability
If real-time monitoring and alerting mechanisms are implemented, then data security is enhanced, but system resource consumption increases
Solution Approach 1:
The system implements partial monitoring by focusing computational resources on users identified as potential anomalies rather than continuously monitoring all users at full depth. This selective approach maintains security by investigating suspicious patterns while reducing overall resource consumption through targeted rather than exhaustive monitoring.
Data Source
AI summary
A system, method, and machine-readable storage medium for detecting an anomaly are provided. In some embodiments, the method includes computing an access rate of a set of entities for each user of a plurality of users. The access rate may refer to data operations for the set of entities stored by a storage system. The method also includes normalizing the access rates for a subset of the plurality of users, the subset belonging to a community. The method further includes determining whether a normalized access rate from among the access rates satisfies a threshold. The method also includes detecting an anomaly in response to a determination that the normalized access rate satisfies the threshold.