In-Memory Bloom Filter Query Processing to Cut Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems face inefficiencies in processing queries due to external processors, leading to increased processing time and power consumption from data movement between memory arrays and processing resources.
Innovation Solution
Implementing in-memory query processing using probabilistic data structures, such as bloom filters, within a PIM capable device to determine which data blocks are immaterial, reducing the need for read operations and enhancing data management efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If external processors are used to process queries, then processing resources are separated from memory, but processing time and power consumption increase due to data movement between memory arrays and processing resources
Solution Approach 1:
The patent combines processing resources with memory arrays by implementing query processing functionality directly within the memory device. The processing circuitry is integrated with the memory array, allowing data to be processed in-place without external processors, thereby eliminating data movement overhead and reducing processing time.
Solution Approach 2:
The patent introduces an address generation circuit as an intermediary component that generates addresses for data blocks based on query parameters. This intermediary enables efficient in-memory query processing by directly accessing relevant data blocks without requiring external processor intervention, thus reducing processing time and power consumption.
2Productivity
If external processors are used to process queries, then processing resources are separated from memory, but power consumption increases due to data movement between memory arrays and processing resources
Solution Approach 1:
The patent merges processing resources with memory arrays by implementing query processing functionality directly within the memory device. The processing circuitry is integrated with the memory array, allowing data to be processed in-place without external processors, thereby eliminating data movement overhead and reducing processing time.
Solution Approach 2:
The patent introduces an address generation circuit as an intermediary component that generates addresses for data blocks based on query parameters. This intermediary enables efficient in-memory query processing by directly accessing relevant data blocks without requiring external processor intervention, thus reducing processing time and power consumption.
3Productivity
If probabilistic data structures are used for query processing, then unnecessary read operations are reduced, but device complexity increases due to implementation of probabilistic data structures within memory
Solution Approach 1:
The patent implements self-service query processing where the memory device autonomously determines which data blocks are immaterial using probabilistic data structures stored within the same device. The processing circuitry uses these structures to filter out irrelevant data blocks without external processor assistance, reducing unnecessary read operations while maintaining manageable complexity through integrated design.
Data Source
AI summary
The present disclosure includes apparatuses and methods related to in-memory query processing using probabilistic data structures. An example device comprises an array of memory cells and logic coupled to the array and configured to receive, from a host, a query related to a dataset stored in the array of memory cells and implement a bloom filter using a hash algorithm and comparison operations to process the query by the memory device without transferring the dataset to the host.