Hardware Vector Processing for Crypto-Digest Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Previous data storage systems face significant limitations in processing host I/O requests due to high CPU resource utilization for calculating crypto-digests, which is CPU-intensive and reduces the rate at which data storage systems can process requests.
Innovation Solution
The technology employs hardware vector processing logic to simultaneously calculate multiple crypto-digests for blocks of host data, reducing the need for pre-allocated context and context manager objects and optimizing CPU usage by performing vector operations on crypto-digest calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional CPU-based crypto-digest calculation is used for data deduplication, then data deduplication functionality is achieved, but CPU resource utilization becomes excessively high (greater than ten percent of all clock cycles)
Solution Approach 1:
The patent replaces traditional CPU-based mechanical calculation with hardware vector processing logic that uses parallelized crypto-digest calculations. The storage processor includes dedicated vector processing units that can simultaneously calculate multiple crypto-digests using hardware acceleration, substituting the sequential CPU-based approach with parallel hardware-based computation to dramatically reduce CPU utilization while maintaining deduplication functionality
Solution Approach 2:
The patent segments the crypto-digest calculation process into multiple parallelizable operations. Instead of calculating one digest at a time using sequential CPU instructions, the system divides the workload into multiple independent digest calculations that can be performed simultaneously by the vector processing logic, reducing the total time and CPU resources required
2Reliability
If sequential CPU-based crypto-digest calculation is used, then data deduplication is performed, but the rate at which host I/O requests can be processed is significantly limited
Solution Approach 1:
The patent enables continuous parallelized crypto-digest calculation across multiple blocks of data. The vector processing logic can simultaneously calculate digests for multiple blocks in parallel, maintaining continuous useful computation without sequential bottlenecks. This allows the system to process I/O requests at a higher rate by keeping the vector processing units continuously occupied with parallel digest calculations rather than sequentially processing one block at a time
Solution Approach 2:
The patent segments host I/O requests into multiple blocks that can be processed in parallel. Each block can have its crypto-digest calculated simultaneously by the vector processing logic, enabling the system to handle multiple I/O operations concurrently rather than sequentially, thereby increasing the overall request processing rate
3Device complexity
If pre-allocated context and context manager objects are used for crypto-digest calculation, then CPU resource allocation is simplified, but processing overhead increases
Solution Approach 1:
The patent extracts the context management overhead from the crypto-digest calculation process. Instead of using pre-allocated context and context manager objects that require setup and teardown operations, the system uses a more streamlined approach where the vector processing logic directly performs digest calculations without the overhead of object allocation and management, reducing processing time while maintaining resource allocation simplicity
Data Source
AI summary
In response to a cache flush event indicating that host data accumulated in a cache of a storage processor of a data storage system is to be flushed to a lower deck file system, an aggregation set of blocks is formed within the cache, and a digest calculation group is selected from within the aggregation set. Hardware vector processing logic is caused to simultaneously calculate crypto-digests from the blocks in the digest calculation group. If one of the resulting crypto-digests matches a previously generated crypto-digest, deduplication is performed that i) causes the lower deck file system to indicate the block of data from which the previously generated crypto-digest was generated and ii) discards the block that corresponds to the matching crypto-digest. Objects required by a digest generation component may be allocated in a just in time manner to avoid having to manage a pool of pre-allocated objects.