Internal Storage Device Striping for Parallel Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face limitations in performance due to the need for manual configuration of striped volumes and lack of support for striping in many operating systems, leading to suboptimal performance in applications like DB applications, Backup applications, and Replication/Cloning applications.
Innovation Solution
A computer-implemented method for internal striping within a subset of slices, where a logical unit is split into data chunks and distributed across multiple physical partitions in a storage grid array, allowing parallel processing and balancing workloads across compute modules without introducing additional metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored on a single storage device, then device complexity is reduced, but data throughput and processing speed are limited
Solution Approach 1:
The patent segments data into multiple data chunks and distributes them across multiple physical partitions within a single storage device. This internal striping allows parallel data access and processing, increasing throughput without requiring complex external striping configurations across multiple devices. The segmentation enables multiple compute modules to process different data chunks simultaneously, achieving high-speed data processing while maintaining a simple single-device architecture.
2Speed
If manual configuration of striped volumes is required, then data throughput can be improved, but ease of operation deteriorates
Solution Approach 1:
The storage device performs internal striping automatically without requiring manual configuration by storage administrators. The system self-manages the distribution of data chunks across physical partitions and the coordination of multiple compute modules, eliminating the need for complex manual setup of striped volumes, consistency groups, and zoning. This self-service approach maintains high data throughput while dramatically simplifying operation for users.
Solution Approach 2:
The patent merges multiple functions including data striping, load balancing across compute modules, and metadata management into a unified internal process within the storage device. This integration eliminates the need for separate manual configurations for each function, allowing the system to achieve high performance through a single automated mechanism rather than multiple complex configuration steps.
3Measurement precision
If additional metadata is introduced for data management, then data tracking and management precision improve, but processing latency increases
Solution Approach 1:
The patent extracts the metadata management overhead from the critical data processing path. By implementing internal striping where data chunks are distributed across physical partitions with implicit tracking, the system eliminates the need for additional metadata structures that would slow down processing. The compute modules can directly access and process data chunks without metadata lookup delays, achieving both precise data management and low latency through this extraction approach.
Data Source
AI summary
A computer-implemented method, according to one embodiment, is for performing internal striping within a subset of slices. The computer-implemented method includes: receiving, by a computer, a logical unit; splitting, by the computer, the logical unit into a plurality of data chunks; and distributing, by the computer, the plurality of data chunks across the subset of slices such that the plurality of data chunks are striped across the subset of slices, and striped across a plurality of physical partitions in each of the subset of slices. Moreover, each of the subset of slices correspond to a different physical storage module in a single storage device. Other systems, methods, and computer program products are described in additional embodiments.