Balancing Primary Placement Groups in Object Storage via OSD Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, unbalanced distribution of primary placement groups across Object-based Storage Devices (OSDs) leads to uneven workload, affecting cluster performance, as existing methods fail to consider the impact of cache disks, resulting in wasted performance.
Innovation Solution
A method and system that calculates the weight of each OSD, determines the number of expected balanced primary placement groups based on OSD weights, and uses a Hash algorithm to identify primary placement groups by comparing base values with a preset value, optimizing system performance by balancing primary placement group distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If primary placement groups are distributed without considering cache disk capacity, then the distribution process is simple, but the workload becomes unbalanced and cluster performance is degraded
Solution Approach 1:
The patent introduces OSD weight as a dynamic parameter that reflects both mechanical disk capacity and cache disk capacity. This parameter change enables the distribution algorithm to consider actual storage capabilities, transforming the simple but ineffective uniform distribution into a weighted distribution that balances workload according to each OSD's real capacity, thereby improving cluster performance without excessive complexity
Solution Approach 2:
The system calculates OSD weights based on actual cache disk usage and mechanical disk capacity, creating a feedback mechanism. This feedback allows the primary placement group distribution to adapt to changing system conditions, ensuring that OSDs with more available capacity receive more primary PGs, thus maintaining balanced workload and optimal performance
2Productivity
If cache disk capacity is not considered in primary placement group distribution, then the calculation method is simple, but cache resources are wasted and system efficiency is reduced
Solution Approach 1:
The patent modifies the OSD weight calculation to include cache disk capacity as a key parameter. By changing the weight formula to incorporate both mechanical disk capacity and cache disk capacity, the system ensures that primary placement groups are distributed according to actual available resources, preventing cache resource waste and improving overall system efficiency
Solution Approach 2:
The system performs preliminary calculation of OSD weights before distributing primary placement groups. This preliminary action ensures that the distribution decision is made with full knowledge of each OSD's capacity characteristics, allowing optimal allocation from the outset and avoiding the need for complex real-time adjustments during operation
3Productivity
If primary placement groups are uniformly distributed across OSDs, then the distribution algorithm is simple, but workload balance is poor and performance is affected
Solution Approach 1:
The patent applies local quality by assigning different weights to different OSDs based on their specific capacity characteristics. Instead of uniform treatment, each OSD receives a customized weight reflecting its actual capacity, allowing the distribution algorithm to allocate primary placement groups locally optimized for each device, thereby achieving workload balance without requiring a universally complex algorithm
Data Source
AI summary
Provided are a method and system for balancing and optimizing a primary placement group, and a device and a storage medium. The method includes: determining a theoretical average value; determining a weight corresponding to each object-based storage device based on an actual capacity occupied by the object-based storage device and the theoretical average value; determining the number of expected balanced primary placement groups based on the total number of current primary placement groups and the weight corresponding to the object-based storage device; obtaining a corresponding base value based on an address of each placement group and an address of the object-based storage device, and determining the placement group as a primary placement group in a case where the base value is less than a preset value; and repeating a previous operation until the number of the determined primary placement groups reaches the number of the expected balanced primary placement groups.