Storage Node Data Distribution for Heterogeneous Capacity Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems fail to efficiently utilize storage nodes of varying capacities, leading to performance deterioration and compromised fault tolerance when writing and reading data, as they do not account for capacity differences, resulting in bottlenecks and reduced performance.
Innovation Solution
The system divides data into multiple fragments, generates parity data for error correction, and strategically assigns these fragments and parity data across storage nodes based on predetermined conditions to ensure balanced distribution and minimize parity data read during retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is written equally to all storage nodes without considering capacity, then writing process is simple, but large capacity storage nodes cannot be fully utilized and become bottlenecks reducing writing performance
Solution Approach 1:
The patent applies local quality by making the data distribution strategy adaptive to local conditions of each storage node. Specifically, the system determines the capacity of each storage node and allocates data accordingly - larger nodes receive more data while smaller nodes receive less. This resolves the contradiction by optimizing writing performance through capacity-aware distribution without requiring complex manual configuration, as the system automatically adapts to each node's characteristics.
2Quantity of substance
If data is relocated to balance capacity utilization, then storage area is used sufficiently, but large capacity storage nodes become bottlenecks reducing read performance
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the data distribution parameters based on the specific capacity parameters of each storage node. The system calculates optimal data allocation ratios considering node capacities and relocates data to achieve balanced utilization. This resolves the contradiction by ensuring that read operations can be distributed efficiently across nodes with appropriate data volumes, preventing bottlenecks while maximizing storage capacity usage.
3Reliability
If parity data is stored intensively in certain storage nodes, then fault tolerance is maintained, but performance deteriorates due to uneven load distribution
Solution Approach 1:
The patent applies asymmetry by creating an asymmetric data and parity distribution pattern that matches the asymmetric capacity characteristics of the storage nodes. Instead of uniform distribution, the system strategically places data and parity data in proportion to each node's capacity. This resolves the contradiction by maintaining fault tolerance through proper parity placement while avoiding performance deterioration, as the asymmetric load distribution aligns with the asymmetric hardware capabilities of different nodes.
Data Source
AI summary
An information processing apparatus can prevent performance deterioration, and maintain fault tolerance, in a storage system having storage nodes of different capacities. The apparatus includes a data writing unit to divide received data into divided data, generate a parity data usable when re-configuring the received data having an error, and write divided data and parity data in storage nodes. The apparatus includes a relocation unit to assign a relocation position of the data based on a predetermined condition and store the data in the assigned storage nodes. The apparatus includes a data reading unit to read the divided data so as not to read parity data stored in the storage nodes by identifying the parity data.