Used Identifier Cache With Bloom Filters for Query Data Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of loading data from disk to memory is time-consuming and resource-intensive, often resulting in unnecessary operations when the requested data is not included in the loaded portion.
Innovation Solution
Implementing a used identifier cache and bloom filters to determine whether the requested data is present in the data slice before loading, using a used identifier cache to manage bloom filters efficiently and minimize resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is loaded from disk to memory for query processing, then query retrieval can be performed, but unnecessary data loading occurs when requested data is not in the loaded portion
Solution Approach 1:
The patent applies preliminary action by creating and storing bloom filters in advance for each data slice. These bloom filters are prepared beforehand to quickly determine whether queried identifiers exist in specific data slices, preventing unnecessary data loading operations and reducing query processing time.
Solution Approach 2:
The patent introduces bloom filters as an intermediary data structure between the query system and the actual data slices. This intermediary allows the system to efficiently check identifier existence without loading entire data slices into memory, thus reducing unnecessary data loading while maintaining accurate query results.
2Reliability
If data is loaded from disk to memory, then data can be retrieved, but additional memory resources are consumed
Solution Approach 1:
The patent extracts only the essential information needed for query filtering by using bloom filters to check identifier existence before loading data. This extraction principle allows the system to determine data presence without loading the actual data content, significantly reducing memory resource consumption while maintaining retrieval capability.
Solution Approach 2:
The patent uses bloom filters as a compact copy or representation of data slice contents. Instead of loading entire data slices into memory, the system uses these space-efficient probabilistic data structures to verify identifier presence, thereby reducing memory usage while preserving data retrieval functionality.
3Measurement precision
If bloom filters are created for all identifiers, then query accuracy is improved, but resource usage increases
Solution Approach 1:
The patent applies local quality by creating bloom filters selectively for specific data slices rather than uniformly for all identifiers. Each data slice has its own bloom filter tailored to its contents, optimizing the balance between query accuracy and resource consumption by focusing computational effort only where needed.
Data Source
AI summary
Embodiments are described for a database management system comprising a memory and at least one processor coupled to the memory. The at least one processor is configured to receive a plurality of queries and determine a first identifier based on the plurality of queries. The at least one processor is further configured to create a first bloom filter based on the first identifier and receive an additional query corresponding to the first identifier. The at least one processor is further configured to execute the first bloom filter.