Computational Storage Devices Accelerate Operations via NIC Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage systems face performance bottlenecks due to bus contention from a large number of solid state disks (SSDs) and conflicting objectives between data protection and optimization techniques, leading to suboptimal data replication schemes.
Innovation Solution
The technology leverages computational network and storage components for local processing, using similarity hashing, systemic erasure coding, Voronoi mapping, and key/value storage to improve resource utilization and accelerate storage operations by routing similar data to the same storage devices and determining erasure-coding set membership based on fragment placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large number of SSDs are connected to a storage system, then storage capacity and data protection are improved, but bus contention increases and performance deteriorates
Solution Approach 1:
The patent divides the storage system into multiple independent storage tiers, each with its own computational resources and processing capabilities. This segmentation allows parallel processing of storage operations across different tiers, reducing bus contention while maintaining data protection through distributed erasure coding across the segmented architecture.
Solution Approach 2:
The patent introduces computational storage devices as intermediaries between the host system and storage media. These intermediaries perform data processing, compression, and erasure coding operations locally, reducing the data transfer burden on the system bus while maintaining data protection capabilities across multiple SSDs.
2Reliability
If traditional data protection techniques like erasure coding are applied, then data reliability is improved, but storage optimization techniques like deduplication become less effective
Solution Approach 1:
The patent implements dynamic coordination between erasure coding and deduplication operations, where the system adaptively adjusts the application of these techniques based on data characteristics and workload conditions. This dynamic approach allows both data protection and storage optimization to work synergistically rather than conflicting with each other.
Solution Approach 2:
The patent changes the parameters and conditions under which erasure coding and deduplication are applied, allowing the system to optimize storage efficiency while maintaining data protection. By adjusting these parameters dynamically, the system can achieve both reliable data protection and effective storage optimization.
3Reliability
If data is spread across multiple storage devices for protection, then fault tolerance is improved, but local access performance deteriorates
Solution Approach 1:
The patent segments data into different types (hot, warm, cold) and stores them in appropriately optimized storage locations. Frequently accessed data can be kept in faster storage media or cached, while maintaining the distributed structure for fault tolerance. This segmentation allows local access performance to be improved without sacrificing fault tolerance.
Solution Approach 2:
The patent applies different storage qualities and performance characteristics to different data segments based on their access patterns and requirements. Hot data receives higher performance optimization with faster access paths, while cold data can be stored in more capacity-efficient locations, all while maintaining the distributed fault-tolerant structure through erasure coding.
Data Source
AI summary
Methods, non-transitory machine readable media, and computing devices that accelerate storage operations are disclosed. With this technology, computational storage devices are assigned by a NIC processor of a computational network interface controller (NIC) to storage locations corresponding to respective partitions of an n-dimensional space. One of the storage locations to which a data location in the n-dimensional space maps is then identified by the NIC processor. The data location is determined based on a hash for data associated with a received storage operation. The NIC processor subsequently communicates via a system bus with a storage processor of one of the computational storage device to service the storage operation. The storage processor is coupled directly to flash media of the one of the computational storage device that is assigned to the one of the storage locations and maintains a key/value store comprising the data.