DBMS Offloading Snippet Scheduling for Computational Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In a disaggregated environment where a database management system (DBMS) and a storage system are separated, the existing methods face challenges with bandwidth scarcity, reduced response speed, and increased power consumption due to the large amount of data transmission during query execution.
Innovation Solution
A method is proposed where the DBMS generates an offloading code, known as a snippet, to offload a part of query computations onto a computational storage drive (CSD). This snippet includes a parsed query and BufferAddress, and is scheduled based on the resource state of the CSDs to optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the DBMS processes all query computations locally, then query execution reliability is maintained, but bandwidth consumption increases and response speed decreases
Solution Approach 1:
The patent segments query computations into two parts: filtering operations executed at the storage level (CSD) and aggregation operations executed at the DBMS level. This segmentation allows the storage system to process large amounts of data locally, reducing bandwidth consumption while maintaining query execution reliability through coordinated processing between storage and DBMS.
Solution Approach 2:
The patent introduces an intermediary mechanism where the storage system (CSD) acts as a mediator between the client and the DBMS. The storage system receives queries, performs filtering operations locally using the offloading code, and returns filtered results to the DBMS for final aggregation. This intermediary approach reduces the data transmission burden on the network while maintaining processing reliability.
2Device complexity
If the DBMS processes all query computations locally, then processing control is simple, but power consumption increases
Solution Approach 1:
The patent segments computational tasks between the storage system and DBMS, with the storage system handling data filtering operations. This segmentation reduces the computational burden on the DBMS and its associated hardware, thereby reducing power consumption while maintaining manageable processing control through the offloading code mechanism.
Solution Approach 2:
The storage system performs self-service by executing filtering operations locally using the offloading code provided by the DBMS. This self-service capability allows the storage system to process data independently without requiring continuous intervention from the DBMS, reducing overall system power consumption while maintaining processing control simplicity.
3Loss of energy
If query computations are offloaded to storage, then bandwidth consumption decreases, but scheduling complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-compiling and storing offloading code at the storage system before query execution. The DBMS generates the offloading code based on query patterns and stores it in advance at the appropriate CSDs. This preliminary preparation eliminates the need for complex real-time scheduling decisions during query execution, reducing scheduling complexity while maintaining low bandwidth consumption.
Solution Approach 2:
The patent uses copying by replicating the same offloading code across multiple CSDs that store the same database tables. Instead of creating unique schedules for each query, the system copies the pre-generated offloading code to the relevant storage locations. This copying approach simplifies scheduling while enabling efficient parallel processing across multiple storage systems, reducing bandwidth consumption.
4Productivity
If multiple storages are used, then data processing capability increases, but resource allocation complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-determining which CSDs should execute which filtering operations based on query patterns and data distribution before actual query execution. The DBMS generates offloading code that specifies the target CSDs in advance. This preliminary allocation strategy enables multiple storages to work in parallel efficiently, increasing data processing capability while avoiding real-time resource allocation complexity.
Data Source
AI summary
There is provided a method for scheduling offloading snippets based on a large amount of DBMS task computation. A DB scheduling method according to an embodiment of the disclosure includes determining, by a DBMS, whether to offload a part of query computations upon receiving a query execution request from a client, generating, by the DBMS, an offloading code which is a code for offloading a part of the query computations, based on the received query, when offloading is determined, selecting one of the plurality of storages in which a DB is established, and delivering the offloading code. Accordingly, snippets which will be generated simultaneously are scheduled for CSDs, so that resources are equally utilized, a query execution time is reduced, and reliability on data processing is enhanced.