Distributed Cache Pool for Block Storage Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The scalability of cache media in block storage-based data processing systems is limited by hardware development, restricting the practical function and performance enhancement in handling large volumes of data.
Innovation Solution
A method and system where multiple storage nodes with independent CPUs and cache media form a cache pool, allowing data operation requests to be processed across nodes, enhancing scalability and performance by distributing caching and processing tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If JBOD cascading is employed to extend the number of disks, then the storage capacity is improved, but the cache medium scalability is limited due to the limited number of SAS channels
Solution Approach 1:
The system segments the storage functionality by separating service processing nodes from storage nodes. Each storage node independently manages its own cache medium and non-volatile storage medium, eliminating the bottleneck of centralized cache management and allowing independent scaling of storage capacity without being constrained by the number of SAS channels on a single service processing node.
Solution Approach 2:
The patent transitions from a single-service-processing-node architecture to a multi-storage-node architecture, adding a dimensional expansion to the system. This allows the cache pool to scale horizontally by adding more storage nodes, rather than being vertically constrained by the SAS channel capacity of a single service processing node.
2Productivity
If the capacity of cache medium in service processing node is increased to support JBOD extension, then the caching performance is improved, but the hardware complexity and cost increase
Solution Approach 1:
The patent merges the cache medium with the storage nodes, creating a distributed cache pool where cache capacity is combined across multiple nodes. This approach achieves high caching performance without concentrating all cache resources in a single service processing node, thereby reducing hardware complexity and cost at any given point while maintaining overall system performance.
Solution Approach 2:
Each storage node is designed to be multi-functional, serving both as a storage unit and a caching unit. This eliminates the need for dedicated cache hardware in service processing nodes, as the storage nodes themselves provide caching services, reducing overall hardware complexity and enabling more flexible resource allocation.
3Adaptability or versatility
If more SAS channels are provided to support direct connection of more disks, then the cache scalability is improved, but the service processing node complexity increases
Solution Approach 1:
The system segments the disk connection functionality across multiple storage nodes, each with its own CPU and cache medium. This segmentation allows the service processing node to maintain a limited number of SAS channels while still supporting extensive storage capacity through the distributed architecture of multiple storage nodes.
Solution Approach 2:
Storage nodes act as intermediaries between the service processing node and the non-volatile storage media. Each storage node receives data operation requests from the service processing node, processes them locally using its own CPU and cache medium, and manages connections to non-volatile storage blocks, thereby reducing the SAS channel burden on the service processing node.
Data Source
AI summary
The present disclosure relates to the field of information technology, and in particular, to a block storage-based data processing method, apparatus, and system. The block storage-based data processing method provided in embodiments of the present disclosure is applied in a system including at least two storage nodes, each storage node including a CPU, a cache medium, and a non-volatile storage medium, and the cache medium in all the storage nodes forming a cache pool. According to the method, after receiving a data operation request sent by a client, a service processing node sends the data operation request to a corresponding storage node in the system according to a logical address carried in the data operation request, so that the data operation request is processed in the cache medium of the storage node under control of the CPU of the storage node.