Block Cache Thread Model for Low-Latency Virtualized I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtualization technologies fail to leverage performance improvements in virtualized environments due to inefficiencies in storage access and management, particularly in virtualization environments.
Innovation Solution
Implement a thread model and algorithm that leverages single producer single consumer (SPSC) submission queues and completion queues and a queue pairs for efficient I/O operations, leveraging single producer multiple consumers (SPMC) and multiple producers single consumer (MPSC) modes to optimize I/O operations in virtualized environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional log-based write cache methods are used with sequential commit or merged logs, then data consistency is maintained, but I/O throughput and queue depth utilization are degraded
Solution Approach 1:
The patent segments the log write operations into multiple independent queues (log queues) that can be processed in parallel. Each queue can be committed independently, allowing multiple consumers to process different queues simultaneously. This segmentation enables the system to achieve both high throughput (by parallel processing) and data consistency (by maintaining ordered commits within each segmented queue).
Solution Approach 2:
The patent introduces a new dimension of parallelism by organizing logs into multiple queues rather than using a single sequential log. This dimensional change from single-queue sequential processing to multi-queue parallel processing allows the system to utilize multiple CPU cores and I/O channels simultaneously, dramatically improving throughput while maintaining consistency through controlled commit semantics in each queue dimension.
2Productivity
If multiple threads are used for de-staging and backend tasks, then I/O operations can be processed in parallel, but CPU contention and resource overhead increase
Solution Approach 1:
The patent implements a self-service mechanism where the system automatically manages thread creation, task distribution, and resource allocation based on current load conditions. The multi-threaded architecture with work queues allows tasks to be dynamically assigned to available threads without centralized scheduling overhead, enabling the system to self-regulate CPU usage while maintaining high I/O processing parallelism.
3Reliability
If log overlap checking is performed to avoid concurrent writes, then data integrity is maintained, but de-staging efficiency is reduced
Solution Approach 1:
The patent segments the log management space into multiple independent queues, each with its own overlap checking and de-staging process. This segmentation allows parallel de-staging operations across different queues while maintaining data integrity within each queue. The segmentation effectively divides the critical section (overlap checking) into smaller, parallelizable units, improving overall de-staging efficiency without sacrificing data integrity.
4Productivity
If queue depth is increased to utilize storage bandwidth, then I/O throughput improves, but latency for individual operations increases
Solution Approach 1:
The patent implements dynamic queue depth management where the system adjusts the number of pending I/O operations in each log queue based on storage device response times and system load conditions. When storage response is fast, queue depth is increased to maximize bandwidth utilization. When latency increases, queue depth is dynamically reduced to maintain low operation latency. This dynamic adjustment allows the system to optimize the trade-off between throughput and latency in real-time.
Data Source
Figure 1
Figure 2A~2D
Figure 2E
AI summary
Requests for a write storage operation are stored in a ring buffer. The write storage operations are executed using polling threads and cache de-stage threads. Dispatchers and worker threads are created for executing the polling threads and cache de-stage threads. Queue pairs for each pair of dispatchers and worker threads are generated. The queue pairs comprise a submission queue and a completion queue. The next available request is retrieved from the ring buffer. A scoring algorithm is used to load balance the queue pairs associated with the worker threads, the scoring algorithm operable to determine a score based a current depth of the submission queue and completion queue.