FPGA-SSD Architecture for In-Storage Query Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for querying large databases are inefficient as they require loading the entire database into memory, displacing other data and discarding most of it after the query, especially when repeated queries are needed.
Innovation Solution
Implementing an FPGA-based SSD architecture that shares a PCIe bus with an SSD controller, using a filter address range to intercept and process acceleration instructions and data locally, reducing the need for data transfer to host memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is loaded into host memory for processing, then query operations can be performed, but memory usage increases and memory displacement occurs
Solution Approach 1:
The patent extracts the data processing function from the host system and places it directly on the SSD controller. The SSD controller now performs filtering and processing operations locally on stored data, eliminating the need to load data into host memory. This extraction of processing capability resolves the contradiction by enabling query operations without increasing memory usage.
Solution Approach 2:
The SSD controller acts as an intermediary between the host and stored data. Instead of the host directly loading data into memory for processing, the SSD controller intercepts processing requests and performs filtering operations locally, returning only relevant results to the host. This intermediary role eliminates memory displacement while maintaining query processing capability.
2Speed
If large databases are loaded into memory for repeated queries, then query efficiency improves, but memory resources are consumed and other data is displaced
Solution Approach 1:
The SSD controller performs preliminary filtering and processing operations directly on the stored database before data is requested by the host. By pre-processing data and identifying relevant records in advance, the system achieves fast query execution without loading entire databases into memory, thus preserving memory capacity for other uses.
Solution Approach 2:
The SSD controller provides self-service processing capabilities, autonomously performing filtering, sorting, and data retrieval operations without requiring host intervention or memory loading. This self-service approach enables repeated queries to execute quickly using only the SSD's local processing resources, maintaining query speed while avoiding memory consumption.
3Productivity
If data processing is performed on the host, then processing capability is sufficient, but data transfer and memory operations consume time and resources
Solution Approach 1:
The patent segments the data processing system into two independent components: the host system responsible for high-level query formulation, and the SSD controller responsible for data retrieval and filtering operations. This segmentation allows processing to occur directly at the storage device, eliminating data transfer time between host and memory while maintaining overall processing capability.
Solution Approach 2:
The SSD controller serves as an intermediary processing layer between the host and stored data. It receives processing requests from the host, performs filtering and data operations locally without transferring data to host memory, and returns results directly to the host. This intermediary approach eliminates data transfer time while preserving full processing capability.
Data Source
AI summary
A system is disclosed. An upstream interface enables communication with a processor; a downstream interface enables communication with a storage device. The system may also include an acceleration module implemented using hardware to execute an acceleration instruction. The storage device may include an endpoint of the storage device for communicating with the acceleration module, a controller to manage operations of the storage device, storage for data, and a storage device acceleration module to assist the acceleration module in executing the acceleration instruction.