Host-Storage Co-Processing for Scan-Filter Task Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing methods, such as using additional compute units like FPGAs or computational storage, face bottlenecks due to large data movement and computational overload, leading to bandwidth issues and inefficient utilization of storage device capabilities.
Innovation Solution
Implementing a host-storage co-processing method that selectively offloads scan-filter operations based on sampling and filtering, determining the saturation of computing resources within the storage device to efficiently distribute data processing tasks between the host and storage device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If additional compute units are used to offload scanning and filtering operations, then computational capability is improved, but data movement increases causing bandwidth bottlenecks
Solution Approach 1:
The patent segments the SQL query processing into two distinct phases: scanning operation (performed by storage device) and filtering operation (performed by host processor). This segmentation allows each component to handle only the necessary portion of data, reducing overall data movement while maintaining computational capability.
Solution Approach 2:
The patent introduces a query processing module in the storage device that acts as an intermediary to perform scanning operations locally. This intermediary processes data at the storage level before transmission, reducing the volume of data that needs to be moved to the host while still providing computational offloading.
2Quantity of substance
If computational storage is used to offload database scanning and filtering operations, then data movement is reduced, but computational capability becomes insufficient causing computational blocking
Solution Approach 1:
The patent divides the filtering operation into two parts: preliminary filtering (performed by storage device) and final filtering (performed by host processor). This segmentation enables the storage device to handle basic filtering tasks, reducing data movement, while the host processor completes the complex filtering operations, ensuring sufficient computational capability.
Solution Approach 2:
The patent implements partial filtering at the storage device level rather than complete filtering. The storage device performs initial filtering to reduce data volume, and the host processor performs additional filtering as needed. This partial action at each level optimizes the balance between data movement reduction and computational capability.
3Measurement precision
If all data is processed by the host processor, then computational accuracy is maintained, but the processor becomes overloaded creating a computational bottleneck
Solution Approach 1:
The patent segments processing tasks by placing scanning operations in the storage device and filtering operations in the host processor. This segmentation offloads repetitive scanning tasks from the host processor, improving throughput, while the host processor maintains control over filtering operations to ensure computational accuracy.
Solution Approach 2:
The storage device performs self-service by executing scanning operations autonomously without requiring host processor intervention. This self-service capability reduces the host processor's workload and improves overall system throughput while maintaining processing accuracy through coordinated operation.
Data Source
AI summary
A host, a storage device, and a data processing method of the host and the storage device are disclosed. The data processing method of a storage device includes receiving a query request involving a scan-filter task from a host, determining whether a computing resource within the storage device is saturated, with respect to the scan-filter task, determining whether the storage device is needed to perform a scan-filter operation, based on determining that the computing resource within the storage device is not saturated, performing the scan-filter operation on raw data within the storage device to obtain processed data, based on determining that the storage device is needed to perform the scan-filter operation and transmitting the processed data to the host.