Probabilistic Eviction of Partial Aggregation Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data aggregation systems, limited memory space in aggregation computers can restrict the types of operations performed, and sending partial results to client computers shifts computational burden, while also increasing transmission and processing loads.
Innovation Solution
Implementing probabilistic eviction of partial aggregation results from constrained results storage, where the likelihood of reoccurrence determines which results to evict, optimizing storage space and reducing transmission and processing burdens on client computers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If partial aggregation results are stored in constrained memory space, then aggregation operations can continue without data loss, but memory space becomes insufficient for all results
Solution Approach 1:
The patent applies probabilistic eviction based on reoccurrence probability parameters to determine which results to evict from constrained memory. By calculating and comparing reoccurrence probability values, the system intelligently manages memory space, evicting results with lower probabilities of reoccurrence while maintaining results with higher reoccurrence probabilities, thus resolving the contradiction between memory constraints and data retention requirements
2Ease of operation
If partial results are sent to client computers, then computational burden is shifted, but transmission and processing loads increase
Solution Approach 1:
The patent implements a feedback mechanism where client computers provide feedback about their processing status and needs. The aggregation computer uses this feedback information to dynamically adjust the eviction strategy, selecting which partial results to evict based on client capabilities and current system state, thereby optimizing the balance between computational burden distribution and transmission/processing load
3Loss of information
If all aggregation results are stored, then complete data is available, but memory constraints prevent storing all results
Solution Approach 1:
The patent changes the parameter of result selection from uniform storage to probabilistic selection based on reoccurrence probability. By assigning and comparing probability values to different aggregation results, the system determines which results to retain and which to evict, maintaining data completeness for high-probability results while discarding low-probability results to accommodate memory constraints
Data Source
AI summary
Probabilistic eviction of partial aggregation results may be implemented for aggregation operations performed using constrained result storage. An aggregation operation request may be received from a client and executed by scanning and applying the aggregation operation to data retrieved by scanning a data store. Partial aggregation results that are generated while executing the aggregation operation may be stored in a result store. If a partial aggregation result is generated when no further storage space in the result store is available, then one or more currently stored partial aggregation results may be evicted according to a reoccurrence probability so that the new partial aggregation result may be stored in the result store. The evicted partial aggregation results may be sent to the client.