Hardware Filter for Selective Data Filtering in Storage Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in data traffic between databases and primary memory due to the extraction of big data, leading to increased data transfer and processing demands.
Innovation Solution
A data storage device equipped with a hardware filter that compares read data with filtering condition data to determine if a selective filtering operation is necessary, reducing data traffic by only communicating filtered data to the primary memory when matching key data is present.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all big data is communicated from the database to the primary memory for extraction, then complete data availability is ensured, but data traffic volume increases significantly
Solution Approach 1:
The patent extracts only the necessary filtered data from the big data set before communicating to primary memory. The filtering operation is performed on the database side, extracting only relevant records that match the query conditions, thereby reducing the volume of data traffic while ensuring complete availability of the required data for extraction operations.
Solution Approach 2:
The filtering operation is performed in advance before data communication to the primary memory. By pre-filtering the big data according to query conditions and generating hint information, the system prepares only the necessary data subsets for subsequent extraction, reducing unnecessary data transfer and improving overall system efficiency.
2Measurement precision
If filtering operation is performed on all big data in primary memory, then accurate data extraction is achieved, but processing time and energy consumption increase
Solution Approach 1:
The patent extracts only the necessary filtered data from the big data set before communicating to primary memory. The filtering operation is performed on the database side, extracting only relevant records that match the query conditions, thereby reducing the volume of data traffic while ensuring complete availability of the required data for extraction operations.
Solution Approach 2:
The filtering operation is performed in advance before data communication to the primary memory. By pre-filtering the big data according to query conditions and generating hint information, the system prepares only the necessary data subsets for subsequent extraction, reducing unnecessary data transfer and improving overall system efficiency.
3Speed
If filtering condition data is stored in hardware filter, then filtering speed is improved, but device complexity increases
Solution Approach 1:
The patent introduces a hardware filter as an intermediary component between the database and primary memory. This hardware filter receives filtering condition data and performs rapid comparison operations on read data, generating hint information that guides subsequent filtering operations. The hardware filter acts as a mediator that accelerates the filtering process while managing the complexity through dedicated hardware functionality.
Data Source
AI summary
A data storage device includes a first memory device configured to provide first read data in response to a first read command, a controller including a hardware filter configured to generate first hint information based on a result of comparison of the first read data with filtering condition data and a processor configured to determine whether the first read data is to be filtered based on the first hint information, selectively filter the first read data based on the filtering condition data based on the determination result, and generate first filtered data, and a second memory device configured to store the first filtered data. The controller communicates the first filtered data to a host.