Access-Pattern Community Evolution via Hash and Entropy Reconciliation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current community detection methods in distributed storage systems fail to account for actual user attributes and do not effectively reconcile communities over time, making it difficult to track the evolution of communities and detect anomalies.
Innovation Solution
A community detection system that resolves candidate communities with stable communities by using hash values and entropy calculations to determine whether a candidate community is a new stable community or part of an existing stable community, based on user access patterns in a distributed storage system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If communities are re-generated periodically without reconciliation, then new communities can be detected based on current access patterns, but it becomes difficult to track the evolution of communities over time and detect anomalies
Solution Approach 1:
The system performs preliminary actions by generating candidate communities from current access patterns and then reconciling them with previously stable communities before finalizing the community structure. This ensures that community evolution history is preserved while maintaining detection accuracy.
Solution Approach 2:
The system uses feedback by comparing newly generated candidate communities with previously stable communities through reconciliation processes. This feedback mechanism allows the system to track community evolution over time while maintaining accurate anomaly detection capabilities.
2Reliability
If communities are reconciled between time periods, then community evolution can be tracked and anomalies detected, but the complexity of reconciling newly-generated communities with previous communities increases
Solution Approach 1:
The reconciliation process is segmented into distinct steps: generating candidate communities from current access patterns, identifying stable communities from previous periods, comparing them through hash values and entropy calculations, and resolving differences. This segmentation reduces the overall complexity of the reconciliation process.
Solution Approach 2:
The system introduces intermediary mechanisms such as hash values and entropy calculations as mediators in the reconciliation process. These intermediaries simplify the comparison between candidate and stable communities by providing quantitative metrics for similarity assessment.
3Ease of manufacture
If current approaches focus only on connections between nodes, then community detection can be performed using graph structures, but actual user attributes and access patterns are not taken into account
Solution Approach 1:
The system changes the parameters used for community detection from purely structural graph connections to include access pattern frequencies and user behavior attributes. By incorporating access frequency and entropy calculations, the system achieves more precise community detection that reflects actual user behavior while maintaining implementation feasibility.
Data Source
AI summary
A system, method, and machine-readable storage medium for resolving a candidate community are provided. In some embodiments, a method includes obtaining a candidate community and a neighbor set for the candidate community, the neighbor set including zero or more stable communities. The method also includes resolving the candidate community as being a new stable community if the neighbor set is empty. The method further includes resolving the candidate community as being part of a matching stable community if a hash value of the candidate community matches a hash value of one or more stable communities included in the neighbor set. The method also includes resolving the candidate community as being a new stable community if an entropy value is greater than a threshold, the entropy value being based on the candidate community and the neighbor set.