DBMS-CSD Query Offloading Interface for Lower Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In query execution processes, the transmission of large amounts of data between a database management system (DBMS) and a storage system leads to bandwidth scarcity, reduced response speed, and increased power consumption, with existing methods lacking effective solutions for offloading query computations to computational storage drives (CSDs).
Innovation Solution
A method is introduced to define a snippet for offloading a part of query computations from a DBMS to a CSD, involving the generation and delivery of an offloading code, with determination of computation distribution between the DBMS and storage, and utilizing an RDMA connection for direct data transfer without CPU intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all query computations are processed by the DBMS, then computation reliability is maintained, but data transmission volume increases leading to bandwidth scarcity and reduced response speed
Solution Approach 1:
The patent divides query computation into two segments: filtering operations are executed at the storage device level, while the DBMS handles higher-level query processing. This segmentation allows filtered data to be transmitted instead of all raw data, reducing transmission volume and improving response speed.
Solution Approach 2:
The patent introduces an intermediary mechanism where the storage device performs preliminary filtering operations before data transmission to the DBMS. This intermediary processing step reduces the amount of data that needs to be transmitted across the network, thereby alleviating bandwidth constraints.
2Use of energy by moving object
If data is transmitted from storage to DBMS for processing, then computation accuracy is ensured, but power consumption increases due to large data transmission
Solution Approach 1:
The patent extracts the filtering computation function from the DBMS and relocates it to the storage device. This extraction eliminates the need to transmit unnecessary data to the DBMS, reducing both power consumption and transmission volume while maintaining computation accuracy through the storage device's local processing capabilities.
3Productivity
If computational storage drive is used for query processing, then data transmission is reduced, but interface complexity increases between DBMS and storage
Solution Approach 1:
The patent implements a universal interface mechanism where the storage device can perform both traditional storage functions and computation functions through a unified communication protocol. The interface supports multiple operation types (filtering, data retrieval, computation) through a single standardized mechanism, reducing the apparent complexity despite the enhanced functionality.
Data Source
AI summary
There is provided an offloading data interfacing method between a DBMS storage engine and a computational storage device. In a query offloading method according to an embodiment of the disclosure, a DBMS may generate an offloading code which is a code for offloading a part of query computations, based on a received query, when a query execution request is received from a client, and may deliver the offloading code to a storage in which a DB is established in a CSD. Accordingly, in a DB system using a CSD, a snippet for offloading a part of query computations may be defined, and a DBMS and a storage are interfaced by using the offloading snippet, so that a guideline for execution of a query by interworking between the CSD-based storage and the DBMS is provided.