Computational Storage Drive Query Snippet Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In database systems, when there is a large amount of data to be scanned, it leads to insufficient bandwidth, reduced response speed, and increased power consumption, as a significant amount of data needs to be delivered from the storage to the database management system (DBMS).
Innovation Solution
The proposed solution involves a method for executing queries by dividing computations, offloading them to computational storage drives (CSDs), and merging results in parallel, while processing inappropriate computations by the DBMS. This includes generating snippets for offloading query computations, scheduling them for CSDs, collecting results, and merging them back into the DBMS.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is scanned and delivered from storage to DBMS, then query processing can be performed, but bandwidth is insufficient and response speed is reduced
Solution Approach 1:
The patent divides query computations into multiple snippets that can be independently executed. Each snippet represents a segmented portion of the overall query processing task, allowing parallel execution across multiple CSDs and reducing the burden on any single data delivery path.
Solution Approach 2:
The patent extracts computation tasks from the DBMS and relocates them to CSDs. By taking out filtering, aggregation, and other computational operations from the central DBMS and executing them at the storage layer, the system reduces the volume of data that needs to be delivered to the DBMS.
2Productivity
If data is delivered from storage to DBMS, then query processing can be performed, but power consumption increases
Solution Approach 1:
The patent extracts computational work from the DBMS and moves it to CSDs. This extraction of computation from the data movement path allows query processing to occur with minimal data delivery, significantly reducing power consumption while maintaining productivity.
Solution Approach 2:
The CSDs perform computational tasks autonomously without requiring data to be moved to the DBMS. Each CSD independently executes assigned snippets and returns results, enabling self-service processing that eliminates unnecessary data movement and associated power consumption.
3Productivity
If query computations are offloaded to CSDs, then processing efficiency improves, but system complexity increases
Solution Approach 1:
The patent implements a universal snippet execution model where CSDs can execute various types of query computations through a common interface. The snippet format and execution mechanism are standardized across different CSDs, allowing diverse computational tasks to be handled by the same infrastructure, thereby managing complexity.
Solution Approach 2:
The patent introduces snippets as an intermediary mechanism between the DBMS and CSDs. This intermediary format standardizes the interface for computation offloading, simplifying the interaction protocol and managing system complexity by providing a uniform method for task distribution and result collection.
4Loss of time
If multiple CSDs process queries in parallel, then response time decreases, but coordination overhead increases
Solution Approach 1:
The patent segments query processing into independent snippets that can be executed in parallel by multiple CSDs. Each snippet is self-contained with all necessary data and instructions, allowing true parallel execution without requiring complex coordination between processing nodes, thus reducing response time while managing coordination overhead.
Data Source
AI summary
There is provided a query execution method in a DB system in which a plurality of CSDs are used as a storage. According to an embodiment, a query execution method includes: generating snippets for offloading a part of query computations for a query received from a client to CSDs; scheduling the generated snippets for the CSDs; collecting results of offloading; and merging the collected results of offloading. Accordingly, by dividing query computations, offloading, and processing in parallel, while processing query computations that are inappropriate for offloading by a DBMS, a query request from a client can be executed effectively and rapidly.