Parallel LUN Initialization Using Worker Threads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network data storage systems face inefficiencies in initializing multiple data storage units during failover or reboot processes, leading to prolonged outage times due to serial processing methods that underutilize system resources and result in increased latency and potential application failures.
Innovation Solution
Implementing a parallel initialization process using multiple worker threads in addition to an administrative thread to independently process initialization tasks for each volume and LUN, thereby utilizing available processing, memory, and I/O capabilities more effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If serial initialization processing is used for multiple LUNs, then system resource utilization is simplified and control is easier, but initialization time increases and productivity decreases
Solution Approach 1:
The initialization process is segmented into multiple independent worker threads, each capable of initializing LUNs independently. The administrative thread divides initialization tasks among multiple worker threads, allowing parallel processing of multiple LUNs simultaneously, thereby significantly reducing total initialization time while maintaining manageable system control through clear thread分工.
Solution Approach 2:
The system transitions from single-threaded sequential initialization to multi-threaded parallel initialization by adding the dimension of concurrency. Multiple worker threads operate simultaneously on different LUNs, transforming the initialization process from a one-dimensional sequential operation to a multi-dimensional parallel operation, thereby improving productivity without proportionally increasing complexity.
2Reliability
If single administrative thread processes each LUN sequentially, then system control and coordination are simplified, but outage time increases and reliability decreases
Solution Approach 1:
The administrative thread performs preliminary task distribution by assigning initialization tasks to multiple worker threads before actual initialization begins. This preliminary parallelization setup allows the system to process multiple LUNs simultaneously from the outset, reducing outage time and improving system availability without compromising control, as the administrative thread maintains oversight of the parallel operations.
Solution Approach 2:
Multiple worker threads continuously initialize different LUNs in parallel without idle waiting periods. While one worker thread processes a LUN, other worker threads simultaneously process other LUNs, ensuring continuous useful action throughout the initialization process. This eliminates the sequential idle time inherent in single-threaded processing, thereby reducing total outage time and improving system availability.
3Productivity
If parallel initialization with multiple worker threads is implemented, then initialization time is reduced and productivity increases, but system complexity and resource management become more challenging
Solution Approach 1:
Multiple worker threads are designed with universal functionality to initialize different LUNs using the same initialization logic and routines. Each worker thread can independently perform the complete initialization sequence for any assigned LUN, eliminating the need for complex thread-specific code paths. This multi-functionality approach increases initialization throughput while keeping thread management relatively simple, as all worker threads operate uniformly under administrative coordination.
4Productivity
If serial LUN initialization is performed, then memory and processing resources are consumed conservatively, but I/O capabilities are underutilized and overall system efficiency decreases
Solution Approach 1:
Multiple worker threads continuously utilize I/O capabilities by simultaneously performing initialization operations on multiple LUNs. Instead of completing one LUN initialization before starting the next, the system maintains continuous I/O activity across multiple parallel operations. This maximizes I/O subsystem utilization and overall system efficiency, while resource consumption increases proportionally to the useful work being performed rather than representing waste.
Data Source
AI summary
This disclosure uses both an administrative thread and multiple worker threads (N) to process the LUN on-lining work in parallel at both the volume level and the LUN level. When the administrative thread receives the message to start the initialization, the administrative thread assigns the work for reading the VTOC information for the LUNs in a volume to one or more worker threads and moves on to perform additional initialization tasks. N worker threads work on N volumes in parallel. These worker threads then independently send messages (e.g., asynchronous messages) to the file system layer, and once the file system layer is done loading the required buffers, the file system layer sends replies back to the administrative thread. The administrative thread then again assigns work to the worker threads to finally bring the LUNs on-line.