Bucketized Multi-Index Data Structures for SSD Cache Deduplication

The patent segments the deduplication and compression operations into distinct phases: hashing operations are performed in parallel on multiple data chunks simultaneously, and the data structure is divided into fixed-size memory regions (buckets) that can be independently managed. This segmentation allows CPU-intensive operations to be distributed and executed efficiently without blocking I/O operations.

The patent performs preliminary hashing of data chunks to generate fingerprints before actual storage operations. The bucketized data structure pre-allocates fixed-size memory regions and establishes mapping relationships in advance, so that when data needs to be stored or retrieved, the system can quickly determine the appropriate bucket without dynamic allocation or complex searches during I/O operations.

The patent creates a universal bucketized data structure where each bucket is designed to handle multiple key types (LBA keys and CA keys) and varying deduplication scenarios. The fixed-size memory regions can dynamically accommodate different ratios of deduplicated to non-deduplicated data without requiring separate structures, making the system adaptable to varying workloads while maintaining efficient memory utilization.

The patent changes the parameter of memory allocation from dynamic to fixed-size regions. By allocating predetermined fixed-size buckets, the system eliminates the need to reallocate or resize memory structures as deduplication ratios change. The buckets can hold varying numbers of keys based on their fixed capacity, allowing the system to adapt to different deduplication scenarios without wasting memory resources on underutilized allocations.