SSD Controller Deduplication Processor
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deduplication methods are inefficient and expensive, consuming significant CPU, memory, and bus resources due to high read and write latencies in SSDs, especially when handling large-scale data storage measured in exabytes, and require numerous expensive PCIe SSDs for signature comparisons.
Innovation Solution
Incorporating a deduplication processor within the SSD controller to perform signature comparisons directly, reducing CPU workload and data transfer, and allowing for the use of less expensive SATA SSDs by utilizing a single binary bit signal for matching signatures, thereby reducing I/O operations and bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If signature comparisons are performed using conventional methods with CPU and memory, then deduplication functionality is achieved, but CPU workload and memory bandwidth consumption increase significantly
Solution Approach 1:
The patent extracts the signature comparison function from the CPU-memory system and relocates it to the SSD controller. The SSD controller reads signature data directly from SSD storage and performs comparisons locally, extracting only the necessary comparison results and sending them back to the CPU. This eliminates the need for continuous CPU intervention and memory bandwidth consumption during signature comparisons.
Solution Approach 2:
The SSD controller acts as an intermediary between the CPU and SSD storage. It receives comparison requests from the CPU, performs the actual signature comparisons using data read from SSD, and returns only the comparison results. This intermediary approach offloads the computationally intensive comparison operations from the CPU while utilizing SSD's native read capabilities.
2Measurement precision
If entire pages are read from SSD to obtain signatures for comparison, then signature retrieval is achieved, but I/O operations and data transfer volume increase unnecessarily
Solution Approach 1:
The patent extracts only the necessary signature data from the SSD pages during read operations. Instead of transferring entire pages (e.g., 16 KB) to obtain a small signature (e.g., 32 bytes), the system reads pages at the SSD controller, extracts only the signature portions needed for comparison, and transfers only these extracted signatures to the CPU or uses them locally for comparison.
Solution Approach 2:
The patent segments the SSD page read operation into two distinct phases: first, reading the entire page at the SSD controller level; second, extracting only the signature portions from the read data. This segmentation allows the bulk read operation to be performed efficiently at the SSD level while the extraction of specific signature elements minimizes the subsequent data transfer volume to the CPU.
3Productivity
If multiple signature servers are deployed to handle high I/O requirements, then signature comparison capacity increases, but system complexity and hardware costs increase
Solution Approach 1:
The patent merges the signature comparison function with the SSD controller, combining storage and processing capabilities into a single integrated unit. This eliminates the need for separate signature servers and their associated I/O infrastructure. The SSD controller performs both data retrieval and signature comparison operations, consolidating what would otherwise require multiple specialized servers into a single integrated system.
4Speed
If PCIe SSDs are used to provide high bandwidth for signature comparisons, then processing speed increases, but hardware costs increase significantly
Solution Approach 1:
The patent extracts the signature comparison operations from the high-bandwidth PCIe SSD path and relocates them to the SSD controller. This allows the use of lower-bandwidth, less expensive SATA SSDs for storage while performing comparisons locally at the controller level. The expensive high-bandwidth interface is eliminated because the system no longer requires transferring large volumes of data between SSD and CPU for comparison operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data deduplication method is executed by a controller for a solid state drive (SSD). The controller receives a signature for a block of data. The controller performs a comparison of the signature and information in a signature library and determines whether or not the signature matches the information. The controller sends a signal that indicates a result of the comparison. If the signature and the information match then the signal has a first value indicating that the block of data is already stored on the SSD; if the signature and the information do not match then the signal has a second value that is different than the first value.